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More and more individuals, both within and outside higher education, are 
creating and using 3D, virtual reality (VR), and augmented reality (AR) tools 
and systems for research and learning. Applications span the full breadth of 
the disciplinary spectrum: 3D models of vanished historical spaces allow ar- 
chaeologists and historians to test theories about their arrangement and use; 
VR environments allow students of anatomy to explore simulations of the 
human body; AR ties digitized archives and artifacts to physical locations, 
merging the past with the present; and 3D printing enables student artists 
and engineers to share prototypes of their designs. 

The essays in 3D/VR in the Academic Library: Emerging Practices and Trends 
capture just some of ways that 3D/VR is already having an impact on the 
creation and transfer of knowledge. Bringing together the insights of notable 
innovators in research, pedagogy, cultural heritage, and science, the volume 
signals an expansion of these technologies within education while simultane- 
ously acknowledging serious challenges. The continued rapid development 
and expansion of 3D/VR may potentially transform learning, but keeping 
up with these changes can quickly overwhelm the capacity of any single 
department or organization. Decision makers face difficult choices as they 
seek to calibrate their investments in related tools and expertise to the priori- 
ties of their institutions. The interdisciplinary appeal of 3D /VR can generate 
productive collaborations among students, faculty, and staff with different 
academic, technical, and professional backgrounds, and contributing to these 
teams can be rewarding. At the same time, this very interdisciplinarity chal- 
lenges accepted structures of authority: Where should this work take place? 
Who should lead it? In what venues should the work be shared with others? 
How should credit be distributed to contributors? 

Other challenges arise from the adaptation of technologies developed for 
profit-driven industries to educational and nonprofit sectors. As VR technolo- 
gies develop with the interests of big commercial players, and as the means 
of encoding 3D data change alongside VR hardware and software, academic 
and independent creators who value broad access, sharing, and sustainability 
must find ways of maintaining 3D/ VR content so that it is susceptible to cri- 
tique, reproducible, and repurposable. 

To help confront all of these challenges, 3D /VR creators badly need the 
support of specialist librarians and archivists. As interdisciplinary meeting 
points; as service providers dedicated to providing the raw materials for 
research and learning; and as organizations dedicated to the collection, pres- 
ervation, and distribution of knowledge, campus libraries already have staff 
with many of the skills needed for the wise application of new technologies 
to research. Expertise in research methodologies, in digital publication, in 
digital library development, in the use of metadata for discovery, in archival 


appraisal—all of these skills are relevant to the creation of sustainable envi- 
ronments in which educational applications of 3D / VR can thrive. 

One major concern—the digital preservation of 3D/VR data—is a com- 
mon thread that ties the pieces of this volume together. The projects described 
by the authors are all quite distinct in their reasons for being and in their 
potential for reuse. Consequently, the preservation priorities for each body of 
work are different. For uses of 3D/VR to reconstruct vanished historical plac- 
es or artifacts, preserving the context for each reconstruction, the motivation 
behind it, and the choices that experts make about the evidence considered 
are essential elements. Without the context, the rigor of the scholarship that 
underlies digital reconstructions can be obscured. In other cases, when re- 
searchers digitally scan contemporary natural and built environments in 3D, 
preserving “raw” scans of these environments in a common, nonproprietary 
format that is discoverable and reusable may be an important consideration. 
Opportunities for 3D capture may come only once, and the captured data can 
become the best surviving record of a place, artifact, or organism that will 
otherwise be lost to time. For works of pure invention, including games, visu- 
al art, and film incorporating 3D/VR, documentation of the creative process 
could prove critical for future artists and historians. 

This volume’s editors draw upon the contributions of each author to out- 
line practical steps forward for collecting institutions. Notably, they encour- 
age librarians and archivists to envision pathways to 3D/VR preservation 
and access that entail collective action across institutions. Tackling the com- 
plexities of digital preservation for 3D data and VR applications can easily 
stretch skills and technical capacity at one organization well beyond its limits, 
so broad consultation will be necessary to inform the choices that each insti- 
tution must make. 

As VR for the home becomes ubiquitous, its uses for research and learn- 
ing will continue to multiply. Documenting the variety of ways in which re- 
searchers, educators, and information professionals use 3D data and VR soft- 
ware should become a priority for academic libraries and archives, if only to 
inform future applications. For example, virtual environments already show 
great potential for research in philosophy, psychology, and neuroscience. In 
his article, “Are We Already Living in Virtual Reality?,”' Joshua Rothman 
describes how experiments with “virtual embodiment” —a particular type of 
VR that allows a person to temporarily “become” someone else—suggest that 
it is possible to dramatically, perhaps permanently, alter people’s perceptions 
of themselves and others. Such experiences hold the potential for tremendous 
beneficence or tremendous harm. Mapping a future for 3D/VR in education 
will require more than sophisticated technical and financial insights; in time, 
drawing appropriate moral and ethical boundaries could prove even more 
vital. A fully nuanced understanding of the histories of 3D/VR and related 
technologies must be available to help future researchers and teachers cope 
with these challenges. 


—Christa Williford 


' The New Yorker, April 2, 2018 
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and virtual reality (VR) are becoming increasingly common 

in research and teaching. Following a period of intense, yet 
short-lived enthusiasm in the 1990s, VR has returned, showing great 
promise as a tool for enhancing scholarship and pedagogy in the 
context of higher education. VR refers to a set of technologies that 
create immersive digital environments, re-creating the perceptions 
of our everyday experiences in order to simulate places and things. 
Typically, this involves providing stereoscopic visual and aural in- 
formation to the eyes and ears, which the system changes as a user 
moves their head, body, and hands. The release of the inexpensive 
Google Cardboard VR viewer in 2014 has generated significant inter- 
est, with more than 10 million viewers having been shipped (Vanian 
2017). The release of affordable, fully functional VR headsets in 2016 
(most notably, the HTC Vive and Oculus Rift headsets) is fueling ex- 
perimentation in a variety of academic contexts, from library-based 
makerspaces and humanities classes (Figueroa 2018) to architecture 
and design programs (Enis 2016) and law schools (Dilbeck 2016). Ex- 
periments in the VR world aim to bring haptic and olfactory forms of 
perceptual information into the mix to create a greater experience 
of perceptual “fidelity,” that is, the closeness of the user’s experi- 
ence of a simulated, virtual world to the user’s ordinary experience 
of the physical world (Bowman and McMahan 2007). 

Throughout this report, we use the term 3D/VR to stress the 
intertwined nature of 3D data/models and VR-related technologies 
(such as augmented reality), the latter of which may be understood 
as 3D viewing and visualization platforms. Although in some ways 
this is an oversimplification, conceptualizing 3D as the content and 
VR as the platform allows us to simultaneously consider the particu- 
lar creation and curation needs of each aspect separately, as well as 
their interdependencies. 


OD antic at (3D) modeling, 3D capture techniques, 
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As 3D and VR tools are introduced into higher education con- 
texts, they enable faculty and students to engage with highly de- 
tailed 3D data—from cultural heritage artifacts to scientific simula- 
tions—in new ways. The creation and use of 3D data have become 
more widely practiced thanks in part to the decreasing costs of the 
hardware and software for capturing and processing 3D data; pho- 
togrammetry, structured light scanning, laser scanning (LiDAR), 
computed tomography (CT) scanning, and other techniques are 
moving out of the lab and finding new applications in a plethora of 
new fields. With 3D and VR technology, a professor may take stu- 
dents on an immersive field trip to Stonehenge, changing the light- 
ing to simulate various phases of solar events; an archaeologist may 
capture 3D scans of an archaeological excavation and share these 
data with a colleague on the other side of the world in the form of an 
immersive virtual exploration of the site; a biochemistry professor 
may explore complex protein structures with students; or a chemi- 
cal engineer may simulate the movement of fluids in various porous 
rock materials.' 

Recent research has demonstrated the educational benefits of 
3D /VR for teaching in a variety of fields, including architecture 
(Angulo 2013; Milovanovic et al. 2017), anthropology (Lischer-Katz, 
Cook, and Boulden 2018), and medicine (Kersten-Oertel, Chen, and 
Collins 2014), to name a few. More generally, 3D / VR has been shown 
to augment spatial analytic skills (Ragan et al. 2013), particularly 
in such areas as big data exploratory analysis (Donalek et al. 2014; 
Van Dam, Laidlaw, and Simpson 2002), design prototyping (Seth, 
Vance, and Oliver 2011), graph analysis (Ware and Mitchell 2005), 
and analysis of volumetric datasets (Laha, Bowman, and Socha 2014; 
Prabhat et al. 2008). In the humanities, 3D / VR has become a popular 
means of producing new, multimedia forms of scholarship, such as 
3D digitization and VR visualization of medieval manuscripts (En- 
dres, forthcoming), as well as making cultural heritage artifacts and 
sites more easily accessible to scholars and the public (Bentkowska- 
Kafel, Denard, and Baker 2016), and forming a “3-D digital heritage 
ecosystem” (Limp et al. 2011). There is also potential for 3D/VR to 
shape the structure and functioning of the academic library of the 
future (Cook and Lischer-Katz, forthcoming) by using VR to host tra- 
ditional library services, such as collection browsing and searching 
(Cook 2018). 

Because 3D / VR technologies are applicable to a number of 
fields, many academic libraries have become sites for cross-disci- 
plinary research and experimentation in 3D and VR. Some digital 


1 Examples of currently available VR applications that support this type of work 
include Stonehenge VR Sandbox (https: / /store.steampowered.com/app/457650/ 
Stonehenge VR_SANDBOX/); Nefertari: Journey to Eternity, which is built on 
photogrammetric scans of Nefertari’s tomb (https:/ /store.steampowered.com/ 
app/861400/Nefertari_Journey_to_Eternity /); Nanome, which enables the 
exploration of chemicals, proteins, and nanotechnology (https: / /store.steampowered. 
com/app/493430/Nanome/); IrisVR, which supports architecture and interior design 
work (https:/ /irisvr.com/); and Oculus Medium, which provides an artistic sculpting 
and modeling space in VR (https: / / www.oculus.com/medium/ ). These are available 
as of October 23, 2018. 
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humanities and digital scholarship centers, media labs, and maker- 
spaces, for example, have embraced the technology, often providing 
access and support for community members interested in exploring 
3D and VR environments for the first time, meeting curricular goals, 
or examining research data. As 3D and VR projects scale up and 
move outside of the specialist disciplines where they have existed for 
decades (e.g., computer science labs, media arts programs), questions 
arise concerning skills development, interdisciplinary collaboration, 
publication, sustainability, preservation, and reuse. In addition to pro- 
viding access to 3D and VR resources, the academic library, already a 
center for collaboration, instruction, research, and collection preserva- 
tion, is well poised to provide leadership in this field. 


Essays in This Report 


The eight essays presented here emerged from talks given at 3D/VR 
Creation and Curation in Higher Education: A Colloquium to Ex- 
plore Standards and Best Practices, a mini-conference that took place 
March 8-9, 2018, at the Bizzell Library at the University of Oklahoma 
in Norman, Oklahoma. Organized by the editors of this volume who, 
at that time, were Council on Library and Information Resources 
(CLIR) postdoctoral fellows working in academic libraries, the event 
was funded by the Alfred P. Sloan Foundation through a CLIR mi- 
crogrant with co-sponsorship from the University of Oklahoma Li- 
braries, University of California Santa Cruz University Library, and 
Temple University Libraries. Although the primary focus on 3D/ 

VR was intentionally narrow in order to maintain a small, intimate 
group, many of the issues that arose also apply to other immersive 
technologies, including augmented reality (AR), mixed reality (MR), 
and extended reality (XR). By its nature, this report is a snapshot, 

a portrait of the varied professional objectives and workflows that 
have developed around 3D/VR at this moment, rather than a com- 
pendium of universal best practices that will be applicable across 
space and time. 

Over the course of a day and a half, the schedule included mod- 
erated discussions by the organizers interspersed with presenta- 
tions by the eight experts, each representing knowledge in one or 
more of the following areas: 3D content creation, VR visualization 
and analysis, 3D / VR-based educational deployment, and 3D/VR 
data curation. The invited speakers were intentionally chosen from 
diverse subject expertise backgrounds in order to stimulate cross- 
disciplinary conversation about how the unique challenges of 3D // 
VR are handled in a variety of circumstances. In addition to the eight 
invited experts, the participants included librarians, administrators, 
faculty, postdoctoral fellows, and content developers. This approach 
enabled the sharing of knowledge between practitioners in different 
communities of practice, thus fostering dialogue and developing ho- 
listic knowledge about these complex and multifaceted technologies. 

By addressing the full 3D/VR lifecycle within a tightly knit 
community of experts and stakeholders, the organizers of this 
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colloquium aimed to identify points of tension and gaps in existing 
practices and knowledge in order to foster common understanding 
for the librarians and digital curators tasked with supporting and 
managing these new data types. In this context, content creators, 
faculty, librarians, digital curators, and preservationists talked across 
their projects, concerns, and needs. In particular, the conversations 
allowed participants to work toward a common understanding of 
when and how to support 3D/VR throughout production, dissemi- 
nation, and archiving workflows for different projects. The presenta- 
tions and moderated discussions bridged language and workflow 
gaps, allowing experts in 3D scanning and VR development to ex- 
change knowledge with experts in project management and digital 
preservation. 

Practitioners are quick to point out that 3D/VR projects do 
not exist in a vacuum, but rather are implemented alongside other 
research and teaching activities already supported by institutes of 
higher education. The first chapter introduces two collaborative 
frameworks for 3D/VR projects in which teaching and research 
become increasingly intertwined. In “Collaborative and Lab-Based 
Approaches to 3D and AR/ VR in the Humanities,” Victoria Szabo 
presents Duke University’s “lab”-based model of collaboration by 
using a common topic or theme of investigation to find shared goals 
among a range of invested departments and stakeholders, including 
libraries. Groups composed of librarians, developers, faculty, and 
students unite with a common goal to produce, teach, share, and 
evaluate research using 3D / VR media. 

The next four chapters focus more closely on the tools and work- 
flows to create and distribute 3D/ VR content, providing varying 
perspectives of how, why, and for whom these projects are created. 
These chapters focus on the unique concerns of digitization technol- 
ogy, model accuracy, artistic intervention, and optimization for shar- 
ing files in different contexts. Working at the University of Virginia, 
Will Rourk probes the distinction between 3D models and 3D data. 
In “3D Cultural Heritage Informatics: Applications to 3D Data Cura- 
tion,” he stresses the importance of thinking of 3D as “dimensional 
data” within cultural heritage projects and introduces a full range of 
3D technology and scholarly outputs: 3D data, 3D prints, VR experi- 
ences, animation, open-access models, etc. He outlines the different 
workflows for the capture technology, research outputs, and preser- 
vation strategies selected, and he describes the library’s role in what 
he calls “3D cultural heritage informatics (3DCHI).” 

Zebulun M. Wood, Albert William, and Andrea Copeland focus 
on 3D/VR as a central project for classroom-based learning, using 
the Media Arts and Sciences classroom at Indiana University—Purdue 
University Indianapolis (IUPUI) as a collaborative space to build the 
Virtual Bethel project. “Virtual Reality for Preservation: Production 
of Virtual Reality Heritage Spaces in the Classroom” describes their 
students’ work on a highly detailed 3D model of a historically signif- 
icant African American church that was being redeveloped in down- 
town Indianapolis. Incorporating historical research, community 


3D/VR Creation and Curation: An Emerging Field of Inquiry 


relationships, 3D data capture, and digital creation, the authors de- 
scribe developing students’ skills, negotiating student group dynam- 
ics, building on existing community relationships, and addressing 
audience concerns and feedback. While their project does not ad- 
dress library participation, their workflow, focus on instruction, and 
use of archival content provide avenues for librarians to consider 
should the question of their participation in similar projects arise. 

In “Using 3D Photogrammetry to Create Open-Access Models 
of Live Animals: 2D and 3D Software Solutions,” Jeremy A. Bot and 
Duncan J. Irschick introduce the importance of animated 3D models 
of individual animals for the Digital Life Project. For this project, re- 
searchers at the University of Massachusetts-Amherst (with partners 
at the University of Oklahoma) developed new photogrammetric 
techniques for capturing accurate 3D data from living organisms, 
including frogs, sharks, turtles, and other animals. Capturing data 
from a living animal prone to unpredictable motion is a difficult pro- 
cedure that requires a digital artist to register and stitch together the 
many images produced in the capture process in order to produce 
a complete model amenable to animation. The complex workflow 
described underscores both the technical mastery and digital inter- 
vention necessary to transform individual scans into a complete, ani- 
mated model and, as the authors argue, makes open-source software 
an ideal choice to preserve and document the 3D model at various 
stages in the creation process. 

“What Happens When You Share 3D Models Online (In 3D)?” 
focuses on the broader dissemination of 3D models online through 
webGL and WebVR. Thomas Flynn introduces the ways in which 
cultural heritage institutions and libraries use platforms such as 
Sketchfab, a commercial platform to share and sell 3D content, to 
reach new audiences. Dissemination, however, is not preservation; 
Flynn notes that Sketchfab is not a repository and that the files it dis- 
seminates for sharing are necessarily derivatives and optimized for 
web browser sharing. The ease of sharing, annotating, and embed- 
ding data, nonetheless, provides the opportunity to engage with new 
and old audiences in unexpected ways and enables the dissemina- 
tion of 3D models to a global community of scholars and enthusiasts. 

The final three chapters address the complexities of sustainabil- 
ity and preservation of 3D/VR projects by introducing three ongoing 
initiatives to promote better standards and workflows across disci- 
plines. In “Building for Tomorrow: Collaborative Development of 
Sustainable Infrastructure for Architectural and Design Documenta- 
tion,” Ann Baird Whiteside reports on work being conducted under 
the auspices of Harvard University Library’s Building for Tomorrow 
project, funded through the Institute of Museum and Library Ser- 
vices (IMLS). This project is looking at the preservation issues related 
to curating architectural and design 3D data, particularly in regard 
to the archives of architects whose materials are increasingly born 
digital. Some common issues in her chapter that may be applicable to 
other academic contexts include the risk of file format obsolescence 
and the long-term accessibility of 3D collections. Whiteside focuses 
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on the future of 3D architectural data, an ongoing concern for in- 
dustry and education alike for at least the past decade, showing the 
enhanced risks of proprietary formats to medium-term access and 
beyond. She also discusses the need to store multiple types of files 
for each architectural project, which is analogous to work done in 
digital humanities and other research where multiple types of media 
files are combined to create complex, interactive scholarly outputs. 
Furthermore, Whiteside underscores that, because of the complex- 
ity of the situation, any solutions to sustainability questions must be 
community-wide, rather than tackled on an institution-by-institu- 
tion, ad hoc basis. Whiteside’s chapter is particularly important in 
considerations of collective action and shared interinstitutional mis- 
sions, since small institutions need to be able to adopt standards and 
best practices, but require the support of larger institutions to do so. 

Jessica Meyerson’s chapter, “3D / VR Preservation: Drawing ona 
Common Agenda for Collective Impact,” introduces concepts from 
her work with the Software Preservation Network that can be ap- 
plied to curating and preserving 3D/VR software into the future. 
While working with 3D/VR requires specialized workflows with 
robust version control, involves a variety of file formats and data 
types with multiple relationships between files, and depends on an 
amalgam of complex hardware and software platforms, Meyerson 
suggests that an examination of 3D/VR curation challenges can be 
guided by findings from other areas of digital curation. She identi- 
fies three major data curation challenges: (1) scale, (2) standards and 
interoperability, and (3) software and hardware dependence. To ad- 
dress these challenges, she argues for a collective impact approach, 
outlines the major components of this approach, and discusses how 
it could be implemented to support the development of standards 
and best practices for 3D/ VR software preservation. 

The final chapter reports on work being conducted by another 
IMLS-funded project, Community Standards for 3D Preservation 
(CS3DP). In “CS3DP: Developing Agreement for 3D Standards and 
Practices Based on Community Needs and Values,” Jennifer Moore, 
Adam Rountrey, and Hannah Scates Kettler review existing projects 
that are tackling the curation problems of 3D/VR and identify gaps 
in these projects that need to be addressed. They argue for more 
consensus-building across different stakeholder groups around pres- 
ervation standards. A critical aspect of creating 3D/VR preservation 
standards is balancing the need to structure workflows around a set 
of common practices with the need to keep new technologies flex- 
ible and open to innovation. Through two national forum meetings 
in 2018 and ongoing collaborative work online, CS3DP brought to- 
gether various stakeholder groups working with 3D data to develop 
standards and best practices. The project formed working groups 
from assembled forum participants to structure stakeholder involve- 
ment and generate consensus on standards and best practices related 
to critical topics, such as metadata and intellectual property rights. 
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Directions Forward 


The issues discussed in this report were chosen to prompt greater 
self-awareness for library professionals as they develop programs 
that use 3D and VR technologies and work to integrate changing 
scholarly demands and conventions with existing library services 
and policies. Across these eight essays, three critical approaches that 
librarians and digital curators need to address as they use 3D /VR 
to support their communities are represented: (1) treat the academic 
outputs that use 3D/VR as scholarly products; (2) build a 3D/VR 
scholarly community to support knowledge exchange across a range 
of stakeholder groups; and (3) develop technical tools, training, and 
infrastructure to support a 3D/VR research ecosystem. 


Treat 3D/VR as Scholarly Products 

Libraries and other institutions need to consider 3D/VR as scholarly 
products in their own right, rather than as illustrations or supple- 
mental material. As such, these projects must be managed through- 
out their research lifecycle like other types of research data. The 
intellectual value of 3D/ VR is still under debate within academic 
communities, but the library can lead the way in establishing the 
products of 3D/VR projects as scholarly outputs, which will encour- 
age greater acceptance of their use in academia. In her chapter, Vic- 
toria Szabo points out that the scholarly standards governing the use 
of 3D/VR are still being developed. While Will Rourk emphasizes 
the need to rethink 3D/VR as data, Szabo argues that 3D/VR should 
also be taken more seriously as scholarly output and should be 
treated alongside the outputs of other types of research projects. Both 
approaches, 3D/VR as data and 3D/VR as scholarly output, are neces- 
sary perspectives and affect the ways in which libraries interact with 
scholars when supporting this type of research. Libraries can aid 

in the acceptance process by supporting peer review practices and 
developing publishing platforms for 3D/VR projects. To this end, 
libraries need to consider how 3D/ VR should fit alongside other 
types of data and scholarly products, which will have an impact on 
scholarly services and research data management programs. 


Build a 3D/VR Scholarly Community to Support 

Knowledge Exchange 

Across the essays in this report there is a collective clarion call to ac- 
tion to build a scholarly community of knowledge- and skill-sharing 
around 3D/VR. The authors collectively ask: How can we—as con- 
tent creators, users, curators, archivists, etc.—prevent the “siloiza- 
tion” of 3D/VR creation and curation practices and knowledge? 
Libraries can play a role in this de-siloization as a cross-disciplinary 
collaborative space. The expertise that libraries offer in data manage- 
ment and digital scholarship practices, together with their ability 

to engage community members, places them in a prime position 

to be partners in 3D/VR scholarship projects. Recent grant-funded 
projects designed to bring communities together and promote 
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knowledge exchange for 3D/VR are leading the way in addressing 
these types of issues. In this report, Ann Baird Whiteside describes 
Harvard University Library’s Building for Tomorrow project, which 
brings together stakeholders in the architecture and design commu- 
nities currently working with 3D data and trying to archive it for the 
long term; Jessica Meyerson lays out the model of the Software Pres- 
ervation Network as a means of structuring knowledge exchange 
across networks of digital curation practice related to 3D/VR; and 
Jennifer Moore, Adam Rountrey, and Hannah Scates Kettler demon- 
strate through the CS3DP project how community-based working 
groups can help establish standards and recommended practices 
that are applicable to both large and small institutions in a variety of 
fields. 

In addition to these innovative, grant-funded initiatives, these 
chapters demonstrate the importance of communicating across com- 
munities to promote effective knowledge exchange.’ This sort of 
cross-disciplinary and cross-institutional communication can take 
the form of forums, white papers, conferences, publications, and the 
like, and they can collectively produce an active scholarly commu- 
nity composed of scholars and information professionals working 
together to develop knowledge in support of these complex technol- 
ogies. Sustained and sustainable collaboration and coordination are 
essential and will help to balance the obligations of content creators 
with those of librarians and digital curators for preserving these new 
research data types and scholarly outputs. Although content creators 
can assist this process by choosing archival formats, capturing meta- 
data, and preparing their materials for archiving throughout the data 
collection process, librarians should take the lead in providing infor- 
mation about standards and best practices and supporting 3D/VR 
creation and preservation workflows. 


Develop Technical Tools, Training, and Infrastructure to 
Support a 3D/VR Ecosystem 

In addition to identifying the need to reconfigure scholarly conven- 
tions and institutional expectations around 3D/VR and enhance 
communication and knowledge exchange across scholarly communi- 
ties and information institutions, the essays in this report stress the 
growing requirements for technical tools, training, and infrastructure 
that are designed specifically to support a holistic 3D/VR research 
ecosystem. The evolution of such an ecosystem must be flexible and 
forward-looking enough to take into account the changing technical 
and scholarly landscape. As libraries are increasingly called upon to 
support knowledge exchange beyond traditional books and journals, 
the creation of novel types of research infrastructure will shape the 


? A third IMLS grant-funded project, Developing Library Strategy for 3D and Virtual 
Reality Collection Development and Reuse (LIB3DVR) is also ongoing, but it was not 
discussed in these proceedings. LIB3DVR will be issuing a series of journal articles 
and a comprehensive white paper on its national forum meetings starting in the 
spring of 2019. More information can be found here: https:/ /lib.vt.edu/research- 
learning /lib3dvr.html. 
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preservation and access expectations of constituents. By being part of 
the conversation, librarians can position themselves to better under- 
stand the needs of the 3D/VR community and the services that the 
library can support. The essays in this volume outline a number of 
approaches by which library professionals can increase their involve- 
ment in 3D/VR. Because every library and academic community 
approaching 3D/VR will be unique, we encourage each one to in- 
corporate an appropriate mix of strategies from across the examples 
provided here. 


Conclusion 


Academic libraries of the twenty-first century have enthusiastically 
embraced digital technology and emergent media as new forms of 
information and data that must be managed and made accessible. 
With greater and greater frequency, the library itself is being seen 
not only as a site of knowledge preservation and access, but also as a 
place for experimentation and knowledge production. Library staff, 
moreover, play increasingly important roles in teaching, research 
support, project collaboration, technology consultation and instruc- 
tion, digital publishing, and more. The future of 3D/VR as a schol- 
arly and pedagogical ecosystem of tools depends on fostering close 
relationships among faculty, librarians, and digital curators. This 
report seeks to elucidate key dimensions of the current landscape of 
3D /VR in academia, in the hope of developing common strategies 
for defining the library’s role and nurturing effective relationships 
between libraries and the range of academic stakeholder groups. 
Developing these new skills and collaborations around emerging 
technologies such as 3D/VR can potentially enhance the profile and 
maintain the relevance of the academic library both as the custodian 
and curator of all forms of research and educational data, and as a 
catalyst for innovation in scholarship and pedagogy at the heart of 
the twenty-first-century university. 
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Chapter 1 


Collaborative and Lab-Based Approaches 
to 3D and VR/AR in the Humanities 


Victoria Szabo 


Abstract 


This paper explores the interdisciplinary humanities lab model for 
collaborative work in 3D and virtual reality / augmented reality (VR/ 
AR). It draws upon our experiences at Duke University with lab 
projects focused on historical and cultural visualization. At their best, 
shared digital projects promote engagement and deeper learning by 
students, expose new research questions for the diverse researchers 
and subject-area specialists involved, and result in applications that 
provide a deeper understanding of historic sites, objects, and phe- 
nomena to the wider public. Working with 3D and VR/ AR in partic- 
ular demands integration of in-depth critical, creative, and technical 
areas of knowledge. Interdisciplinary lab projects leverage the com- 
bined expertise and skill sets of subject-area faculty, librarians, tech- 
nical staff, and students from various academic backgrounds. At the 
same time, however, such an ensemble approach to scholarship may 
disrupt the existing academic ecosystem, challenging disciplinary 
boundaries as well as institutional norms around teaching, research, 
labor, and resource allocation. By honoring diverse stakeholder 
goals, fostering critical conversations around theoretical and tech- 
nical questions, and establishing standards for tools and methods 
beyond the individual case, lab partners from various institutions 
may be able to contribute both to their specific projects and to the 
advancement of sustainable and scalable approaches to future work 
in 3D and VR/AR at their own institutions and beyond. 
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Introduction 


cholars have begun to look to computationally generated 3D 

models, animations, virtual reality, augmented reality, and 

games to represent present, past, future, and fictive environ- 
ments and objects. Complementing textual, image-based, and quan- 
titative analyses with 3D visualizations is nothing new. Precursor 
technologies include illustrative 3D drawings, wooden architectural 
models, annotated exhibitions, immersive dioramas and tableaux, 
and even historical re-enactments. The study and creation of online 
texts, image collections, and time-based media archives is common- 
place. Geographic information systems (GIS) and web mapping are 
supported in classrooms, libraries, and digital humanities centers as 
part of a broader spatial turn in humanistic teaching and research. 
Data visualization has experienced similar growth. Nonspecialists 
may also explore 3D and virtual reality /augmented reality (VR/AR) 
for conducting and communicating their teaching and research. 

Nonetheless, the academic status of digital scholarship itself 
often remains ambiguous. Best practices for production and evalua- 
tion within disciplinary fields are still needed. Evaluation guidelines 
note benefits such as deeper student engagement with course materi- 
als or public access to scholarship. However, undertaking a digital 
project may—or may not—translate into creditable research activity. 
A digital experiment at times may fail, or a tool may fall out of favor 
or reach. This uncertainty can have a chilling effect on scholars who 
might advance the field. 

The project-based humanities lab model of research and teaching 
offers a way to address the gap between discipline-focused objectives 
and digital expertise. In the lab model, participants from various 
backgrounds focus on a common academic question over a sustained 
period. The lab model complements other institutional structures by 
creating novel opportunities for faculty, staff, and students to col- 
laborate across disciplinary bounds. At Duke University’s Wired! 
Lab for Digital Art History & Visual Culture’ and related initiatives, 
for example, first-year curricular experiments with 3D and VR/AR 
in the humanities led to the establishment of new graduate programs 
and to interdisciplinary and international collaborations around best 
practices for future research. 


3D and VR/AR in Historical and 
Cultural Visualization 


Why use 3D and VR/AR? For the Wired! Lab, possible objectives 
might include creating historical reconstructions of significant spaces 
and structures, imagining fictive spaces, annotating or recontextual- 
izing objects of interest, exploring databases and networks across 
multiple dimensions, translating virtual archives into virtual mu- 
seums, or creating virtual settings for the exploration of spatiality, 


Thttp:/ /dukewired.org 
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proxemics, and movements across space and time. We describe this 
activity more generally as “historical and cultural visualization.” 
Students learn about these topics in first-year seminars focused on 
art history topics, in thematic electives, and through participation 

in digital cultural heritage and museum exhibition projects. Some 

of our most advanced work in Wired! has taken place as part of the 
international Visualizing Venice group of researchers (Lanzoni, Gior- 
dano, and Bruzelius 2018). 

The key concept that ties our digital humanities approaches 
together, and especially 3D and VR/AR, is the idea of the model. 
Whether we are modeling an object, an environment, or both, a 
conceptual model underlies our processes and informs what we 
develop. If we add to that a dimension of time or interactivity, we 
make use of that model in a process of simulation or virtualization. 
Key challenges for us in historical and cultural modeling in particu- 
lar may include representing change over time, representing uncer- 
tainty, documenting process and underlying data, producing content 
collaboratively, allowing for counterfactuals and conflicting interpre- 
tations among experts, distinguishing between evidence-based and 
placeholder elements, and connecting to existing or future data and 
systems. And all of this is before we populate the models, program 
in any procedural interactivity, or build in any agents. 

Such challenges exist in any kind of model—3D, digital, or other- 
wise—but are brought into greater relief when working with compu- 
tationally produced 3D and VR systems. The same dimensional, rep- 
resentational qualities that provide greater immersive potential in 3D 
and VR systems also have a greater likelihood of being fudged in an 
attempt to produce a coherent whole, as Will Rourk demonstrates in 
his discussion of 3D data versus 3D models (see Chapter 2). Indeed, 
the software often demands filling in informational gaps, etc. Added 
to these challenges is a relatively high technology turnover rate in 
the field. Without deep understanding of the medium and critical 
engagement with its strengths and weakness, we end up in danger of 
providing what the most strident critics of 3D and VR abhor—inac- 
curate, expensive infotainment that does little to advance or commu- 
nicate knowledge in the field. Providing data transparency, modular- 
ity, and iterability becomes important to counterbalance these effects. 
Creating a black box object in an expensive walled garden, a closed 
digital ecosystem, is also an almost certain way to limit the life span 
and impact of a 3D or VR/AR project over the long term. Therefore, 
scholars in the field need to understand what goes into these models 
and to be active partners in their creation and care. 


The Digital Humanities Ecosystem 


How then do we address the challenges of working with such com- 
plex technologies, while at the same time attending to disciplinary 
concerns? Within many academic libraries, digital scholarship has 
become an important area for partnership and support. As Joan K. 
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Lippincott and Diane Goldenberg-Hart have noted, library support 
of digital scholarship extends beyond digital humanities project sup- 
port to include science and social science research. It also delves into 
broader infrastructure and training needs (Lippincott and Golden- 
berg-Hart 2014). Because 3D and VR/ AR are such cross-disciplinary 
practices, academic libraries may be the ideal place to facilitate 
communication around their critical and creative use. At Duke, for 
example, interest in 3D and VR/AR exists in our engineering pro- 
grams, computer science, campus information technology (IT), and 
the Innovation and Entrepreneurship program, as well as in the arts 
and humanities. 3D/VR also overlaps with game studies, medical 
education, civic awareness, and social justice. Many participants in 
various programs wish to use new media forms for studying, creat- 
ing, and archiving, and they turn to the library for guidance and sup- 
port. Even grant-funding agencies in the humanities now expect rich 
data, access, and sharing plans from the grantees. The need to find 
scalable solutions is becoming more acute. This need includes places 
for storing both working files and final presentational outputs, as 
Ann Baird Whiteside notes in Chapter 6. 

Despite the good will and interest of the constituencies involved, 
and perhaps because the challenges are so complex and overlap- 
ping, it is sometimes unclear in any moment who in the university 
setting should be responsible for providing not only content storage, 
archiving, and production server access, but also instruction and 
training. Is it reasonable to expect librarians to take on this role? Or 
should the staff of departments and programs arrange to teach and 
support digital humanities methods? Perhaps teachers in writing 
programs could take on some responsibility for digital literacies and 
training. But how far should that responsibility go in the case of tools 
and methods that require a significant effort to master? 

Regardless of who teaches them, digital tools and methods are 
sometimes treated as a set of skills to be acquired separately from 
their application to a disciplinary research challenge. Critical atten- 
tion to underlying data structures, metadata standards, the assump- 
tions of software packages, or the affordances of interactive media 
forms may seem a concern better suited to information science, 
media studies, or communication studies, not to mention geography, 
architecture, or media arts. More transparent software interfaces may 
seem to obviate the need for such extra-domain expertise. With their 
user-friendly interfaces and starter tutorials, they may yield more 
impressive outputs than ever before. Yet, their uninformed use may 
produce more critically questionable results. Digital projects that 
force consistency of data, elide uncertainty, cherry-pick examples, or 
eschew complexity and nuance are rightly critiqued. Critics might 
rightly argue that the alluring technologies of 3D and VR/ AR do 
more harm than good. 

The pedagogical challenge does not lie only in the fact that digi- 
tal tools are being taught separately from specific projects, although 
students may find it harder to learn from examples too far afield 
from their disciplinary interest. A bigger issue lies in the follow-up 
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to the initial workshop session or tutorial video, when students (and 
researchers) apply what they have learned. Individual project sup- 
port becomes more difficult to sustain at scale. This is where a proj- 
ect-based lab program can be helpful. Especially when undertaken 
with an eye toward developing models and precedents, in-depth col- 
laboration on specific projects can yield more generalizable results. 
Diverse campus partners may collaborate on such an effort without 
having to commit to generalized support for everyone who attends a 
workshop. The lab model does not replace the library digital scholar- 
ship center. Rather, project-focused labs invite partners from libraries 
and information technology organizations to help create generaliz- 
able solutions and best practices that fit the scholarly questions at the 
heart of the lab’s mission. 


New Directions in Multimodal 
Scholarly Publishing 


Today’s digital humanities ecosystem produces and supports both 
print publications and digital scholarship. The Scholarly Com- 
munications Institute made digital scholarship an explicit focus of 
the 2015 Institute in Chapel Hill, North Carolina, for example.? The 
challenges to the norms of scholarly publishing are already clear. 
The next generation of scholars will be increasingly hybrid, having 
grown up with multimodal technologies throughout their schooling. 
More focus on rigorous peer review, rather than the medium of circu- 
lation, opens up possibilities for new platforms (Modern Language 
Association 2012; American Historical Association 2015; College Art 
Association and Society of Architectural Historians 2016). Online 
publication opens the door to other formats, both as supplements 
and as stand-alone resources. Digital does not necessarily challenge 
the primacy of textual exposition as the primary communicative 
mode of expression for many fields, as Edward L. Ayers (2013) notes. 
In the last several years that has begun to change, with consequences 
for our publishing systems. 

With the support of the National Endowment for the Humanities 
Office of Digital Humanities, the Getty Foundation, The Andrew W. 
Mellon Foundation, and others, gaps between the form of the origi- 
nal production and its documentation are decreasing, and expecta- 
tions are rising for the quality and accessibility of the work present- 
ed—including that in 3D and VR/AR. As it becomes more important 
to review projects in their original media forms, old substitutes for 
original work may no longer be accepted. Digital mapping projects 
may need geoservers to store and share vector and raster data, and 
presentation layers and models, but the support is uneven. Today in 
many cases scholars are still reduced to creating screenshots or video 
documentation of their VR/ AR experiences, at least for archival pur- 
poses. Advances in WebVR and open 3D formats get us closer to be- 
ing able to share such content in its “native” form more openly and 
transparently. Emulators and code repositories also hold promise. 


* https:/ / trianglesci.org /2015-institute / validating-and-valuing-digital-scholarship / 
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In the absence of other solutions, 3D and VR/ AR experiences 
are likely to be served up from common commercial or open-source 
server platforms. Platforms like Sketchfab and the availability of 
Unity3D and Unreal as free presentation environments make it pos- 
sible to share work in a format closer to the original than was pos- 
sible before. Externally hosted digital objects appear as iframes or 
other types of embedded web objects. They have anticipated active 
lifetimes or what may be considered limited-time performances of 
technology exhibitions. The most lasting documentation may para- 
doxically be not in scholarly journals or websites but on social me- 
dia and other generalist platforms like YouTube and Flickr. Valve's 
Steam Store is now allowing almost all content on its game store, 
for example, which may make it easier for some developers to share 
their work. Project code is often shared on GitHub, with GitHub 
Education paving the way for commercial options. It is up to practi- 
tioners in the field to think through the implications of these choices. 

Whatever systems are put in place, whether homegrown, open 
source, commercial, or some combination of these, they should meet 
the development, collaboration, and sharing needs of scholars and 
researchers. Labs can be a great place to test possibilities. A lab with 
a research agenda can serve as a client or real-life testing ground for 
a proposed solution in a way that an internal test may not because 
the end products ultimately need to serve the scholarly community 
for which they were produced. 


Labs in Series and in Parallel 


An individual lab may operate within a larger network of labs, as 
well as within the context of university-wide curricular and research 
programs. Duke’s Wired! Lab was formed in 2009 within the Depart- 
ment of Art, Art History and Visual Studies, with support from Infor- 
mation Science + Studies. It started out with an experimental class of 
five faculty and eight students who were interested in 3D modeling 
and the reconstruction of historical buildings and environments. The 
initiatives in the Wired! Lab today evolved from those beginnings 
and fall within one of two main research categories: Digital Cities / 
Urban Histories and the Lives of Things (Wired! 2018). Current 
Wired! Lab collaborators include faculty in art and architectural his- 
tory, archaeology, visual and media studies; the staff director of the 
departmental Visual Media Center; the university subject-area librar- 
ian for visual studies; and a dedicated digital humanities specialist. 
We also work closely with the Nasher Museum of Art staff and the 
Rubenstein Library’s Special Collections librarians on special proj- 
ects. The group holds regular meetings and Friday afternoon project 
work sessions. 

The disciplinary orientation of the Wired! Lab has been central to 
its high profile in art history circles. Wired! has embraced the “digital 
art history” label and focus for its work. It was founded by Caroline 
Bruzelius, an eminent art historian focused on medieval architecture 
with an interest in 3D modeling of cathedrals, and is now led by Paul 
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Jaskot, also a noted art historian who works on Holocaust architec- 
ture and geospatial analysis. These credits give Wired! academic 
clout. Undergraduates from a wide range of disciplines, as well as 
students pursuing MA, MFA, and PhD degrees, join lab teams as 
participants in lab-based courses and projects organized under the 
digital art history rubric. Lab projects take the forms of posters, pa- 
pers, publications, exhibitions, apps, and VR experiences. 

Wired! does not exist in isolation, however, nor could it achieve 
all it has without benefiting from the existence of other labs and 
programs. In 2011, just as the Wired! Lab was becoming more estab- 
lished, the GreaterThanGames: Transmedia Applications, Virtual 
Worlds, and Digital Storytelling Lab (GTG) was formed at the John 
Hope Franklin Humanities Institute. Like Wired!, GTG was built on 
a curricular experiment. In this case it was the Virtual Realities first- 
year course cluster, which ran from 2007 to 2009 and included cours- 
es from computer science, math, media studies, and classics. The 
game lab that followed made a substantive alternate reality game; 
held a semester-long mobile app development workshop attended 
by students, faculty, and staff; and developed art games in Unity and 
location-based AR apps. It also partnered with Wired! on an architec- 
tural history iPad app, Visualizing San Giovanni e Paolo. Although 
the lab disbanded in 2013, the teaching and research collaborations 
continued. My own work on augmented reality for digital city appli- 
cations grew out of this experience (Szabo 2018). 

Additional labs co-located with Wired! today include the Duke 
Art, Law and Markets Initiative (DALMI),’ the Emergence Lab,’ the 
DiG Digital Archaeology Lab, the Information Science + Studies Lab, 
the John Hope Franklin Humanities Institute Labs,’ and the shorter- 
term Bass Connections’ projects. We also work closely with the Vi- 
sualization and Interactive Systems (VIS) Group, a cross-functional, 
cross-campus interest group/lab that includes the leaders of the 
Duke immersive Virtual Environment (DiVE)’ in the Pratt School 
of Engineering, as well as library representatives focused on digital 
scholarship, visualization, and GIS. Undergraduate, MA, and PhD 
students in Digital Art History and Computational Media, Arts & 
Cultures’ draw upon this whole network in writing their hybrid the- 
ses and dissertations. 


Sharing Solutions 


The key to collaboration lies in finding common ground around re- 
search topics of mutual interest. For example, in Wired! we became 
very interested in creating richly annotated, immersive architectural 
models that could be explored in both active and passive modes. 


3 https:/ / www.dukedalmi.org 

* https: / / cmac.duke.edu/ labs / emergence-lab 
5 https: / / fhi.duke.edu/labs 

6 https: / / bassconnections.duke.edu 

"http: / /virtualreality.duke.edu 

8 http: / /cmac.duke.edu 
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The interaction scripts developed there could be used in other con- 
texts, such as in producing VR art projects and instructional content, 
but they were also useful for diverse other applications in the DiVE 
itself. 

This possibility of sharing code builds upon a choice we made 
several years ago when we discussed the need for shared 3D work- 
flows and pipelines, both for the purposes of introductory instruc- 
tion and for more advanced VR implementation. We collectively 
embraced the decision to standardize on a Unity3D-based workflow, 
investing together in some professional licenses to make it work. 
Our shared goal was to make it easier to move from 3D model to im- 
mersive environment by providing a way for researchers to develop 
and display their own content in Unity3D, and then to export it to 
various platforms, including but not limited to the DiVE. Critically 
for our partners in engineering, our projects become interesting case 
studies for them, too, in terms of user interface design, graphics stan- 
dards, and multimodal data management schemes. From our per- 
spective, standardizing in Unity3D has made it easier for us to move 
into head-mounted display variations on immersive VR experiences 
originally conceived of as limited to CAVE-based installations only. 
We have also begun working with Unity3D plus Vuforia to create AR 
projects focused on generating 3D models from 2D images. 

Current work with Wired! Lab postdoctoral researchers from 
Padua, Italy, who are working on visualizing the Scrovegni Chapel 
Giotto frescoes is an excellent example of how our “new” workflow 
is working across projects and teams. The historical researchers 
were interested in both representing the changing architecture of the 
building itself through interactive building information modeling 
(BIM) and in AR exploration of the frescoes contained within it. The 
DiVE researchers were interested in comparing VR/ AR approaches 
with mobile-based approaches to the same materials for research 
purposes. The Padua team has begun to bring its existing Unity3D- 
based models into the DiVE and head-mounted displays. They are 
also exploring ways to layer information about the frescoes into a 
real-time AR tool for use by visitors. This research may not only 
benefit future projects with our local museum, but also lead to tutori- 
als and platform choices to share with the other labs, library, and IT 
partners (Giordano et al. 2018). 


Curricular Challenges 


As anyone who works in 3D and VR/AR knows, integrating these 
technologies into teaching and research can be complicated when 
working with beginners, whether those beginners are first-year stu- 
dents or advanced faculty. One way to scaffold learning is to use our 
own projects as teaching examples, as the Padua example suggests, 
deconstructing them into tutorial sample files. Another is to create 
lab projects that students can participate in at different levels of the 
curriculum, and to create courses around projects that have a life 
beyond a specific class. For example, we have used the Duke campus 
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and Durham, North Carolina, community resources in both teaching 
and lab research. 

Nonetheless, managing the balance between subject-area and 
technical instruction remains difficult. Regardless of who teaches 
technical topics, we wonder how and when to include tutorials as 
part of the core requirements of a course. We ask if it is better to 
separate technical instruction from primary classwork, and what the 
costs and benefits are of having students working in groups. We also 
ask whether we should privilege historical research over digital ex- 
ecution in final projects. Since many students from computer science 
and engineering are attracted to our courses, we debate whether we 
should allow them to flex their skills or expect them to learn how to 
do qualitative research. By having students work in cross-functional 
teams, we try to have them do a bit of both. This approach also pre- 
pares them for future roles in labs. These are areas of needed future 
clarification and research. Ultimately, the answers may be highly 
context-driven (in terms of student interest and skill level, discipline, 
course subject matter, and particular research questions). 

Despite these efforts, the dream of perfect hybridity in teaching 
with digital technologies is elusive. While the use of some tools, such 
as readily accessible web-based presentation tools, is easy to teach in 
a single training session, more involved GIS, 3D, and VR/AR topics 
require greater degrees of scaffolding and explication. We have also 
created downloadable tutorials and some online instruction, and we 
are working with the libraries to develop more.’ Despite our best 
efforts to pack everything into a single course, we have determined 
that some topics, such as historical GIS, Unity 3D interaction design, 
physical computing, and web-based multimedia communications 
require their own courses for any depth of study. 


Hybrid Futures 


While a few of our faculty, such as Edward Triplett, former CLIR Fel- 
low and now a departmental lecturer and Wired! Lab member, can 
move easily between historical research and technical wizardry, most 
faculty members are more accomplished in one area or another. We 
look to the next generation of scholars for more flexibility and for 
more collaborative teaching and research models than is currently 
the norm. 

To produce this next generation of hybrid scholars, we invite 
graduate students into the labs to serve as research assistants to 
faculty and mentors to undergraduates. We also encourage them 
to develop projects of their own within the PhD Lab in Digital 
Knowledge," a lab I co-direct with Philip Stern from the Depart- 
ment of History, as part of the Duke Digital Humanities Initiative at 
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the John Hope Franklin Humanities Institute." The PhD Lab Schol- 
ars program currently includes about 20 digitally curious gradu- 

ate students from a wide range of humanities disciplines. We meet 
biweekly to share project work, and in between those sessions, we 
meet with technology-focused working groups organized by faculty, 
staff, and library partners. This year’s working groups will focus on 
digital pedagogy, VR/AR, digital mapping, text analysis, and envi- 
ronmental humanities. We also invite our graduate students’ faculty 
mentors to come and see what we are doing, offer critiques, and per- 
haps learn a bit themselves about what is possible. 

The Duke Digital Humanities Initiative also supports a Digital 
Humanities Fellows Program at North Carolina Central University 
(NCCU), a distinguished historically black university with a strong 
community presence. The NCCU Fellows meet monthly at Duke and 
have established their own Digital Humanities Lab at their home in- 
stitution. In the coming year, the faculty from NCCU will work with 
the Information Science + Studies and Wired! Labs on a local neigh- 
borhood history project using location-based 3D and AR to share 
archival materials and to document resident experiences. We hope 
this partnership will contribute both to positive community relations 
and to best practices for public-facing, accessible digital humanities 
research. 


National and International Connections 


These types of digital humanities projects benefit greatly from na- 
tional and international connections. These connections include the 
Wired! Lab partnership in the Visualizing Venice group on research 
and training.” Visualizing Venice started out as an export of the 
“lab” model to our research partners in Venice and Padua. The group 
has multiple collective aims: to advance art history research, to ad- 
vance training, and to explore new modalities of presentation and 
exhibition. From 2011 to 2016, among our other activities, the Visual- 
izing Venice group taught thematically focused, hands-on, two-week 
summer workshops in Venice for junior scholars and young scholars 
interested in digital art and architectural history. The workshops al- 
lowed us to train the next generation of scholars on an international 
level and to reflect together on the next stages for our collaborations. 
Then in June 2018, with the encouragement of the Getty Founda- 
tion, we shifted focus to offer instead an Advanced Topics in Digital 
Art History: 3D Geospatial Networks Institute (#4 DAHVenice2018). 
Unlike our earlier workshops, which were targeted toward individu- 
als, this opportunity privileged applications from interdisciplinary 
teams, building on our experiences with the lab model and extend- 
ing it outward. Teams from 11 countries converged in Venice to share 
projects and ideas, reflecting a wide range of interests. Participants 
include architectural historians, art historians, and archaeologists, 
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as well as experts in GIS, 3D, BIM, and VR, with projects coming 
from various scales and perspectives. The project continues over the 
course of the year. We hope to establish a robust virtual community 
(or lab) in which we produce best practices, grant proposals, shared 
software resources, and joint publications. As organizers, we have 
already benefited from the group’s counsel as we expand our project 
conceptually from the Visualizing Venice focus to the Visualizing 
Cities project. When we next meet as a group, in June 2019, we will 
workshop our deliverables in anticipation of public dissemination. 
At the same time as the Wired! Lab and Visualizing Venice have 
pushed the boundaries of digital art history, we have also furthered 
the conversation about 3D and VR/AR as digital humanities meth- 
ods. With the support of the National Endowment for the Humani- 
ties, Duke University hosted the two-week Virtual and Augmented 
Digital Humanities Institute’ in July 2018. Here our emphasis was 
specifically on the affordances of VR/ AR as media forms, as noted 
in the project description. Participants from the disciplines of art his- 
tory, literature, education, modern languages, history, media art, and 
classics met alongside engineers and developers, including our own 
VIS Group members, as well as local partners in key library units 
focused on mapping, visualization, and digital scholarship. A few 
local faculty who were digital humanities skeptics were also invited. 
The group discussed questions of ethics, longevity, the digital divide, 
access, and fair use, as well as favorite software, best practices for 
development, and project demonstrations. As with the Digital Art 
History group, we are seeking to generalize the lessons learned from 
particular projects into wider principles and best practices, while 
at the same time building a virtual community around our shared 
interests. This push and pull between media possibilities and disci- 
plinary objectives, between common dreams and individual objec- 
tives, between open-ended possibilities and established values, will 
continue to animate our conversations over the coming months and 
years around 3D and VR/AR in humanities scholarship and beyond. 
Within the Wired! Lab, and in other labs like GreaterThanGames 
and the PhD Lab in Digital Knowledge, we have found that what 
ultimately makes the humanities lab work is the participants’ will- 
ingness to communicate; to share time, expertise, and resources; and 
to honor both individual and group project goals. For some partici- 
pants, the hook is a scholarly question; for others, it may be about 
media, tools, systems, pedagogy, or publishing. Beyond the project 
focus that keeps everyone engaged, the secret of the lab model is 
that any self-identified working group can declare themselves a 
lab and behave accordingly. We had participants in Venice declare 
themselves a “lab” with laughter, but also a bit of elation. However 
empowering such a statement may be, the lab model also depends 
on the centralized infrastructure and vision of the libraries, depart- 
ments, foundations, commercial entities, and community partners 
on which it relies. For 3D and VR/ AR in the humanities—in all their 
complexity, expense, and potential—it will take all of us to advance 
the field. 
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3D Cultural Heritage Informatics: 
Applications to 3D Data Curation 


Will Rourk 


Abstract 


The physical elements of history directly link us with the heritage of 
diverse cultures. Artifacts, architecture, and historic sites, active or 
derelict, must be documented while they are extant; otherwise, the 
opportunity to maintain a connection with their historic narratives 
may be compromised or lost to time. Data collection technologies 
such as laser scanning, photogrammetry, and other three dimension- 
al data recording techniques and devices allow the documentation of 
the conditions of a historic place or object with submillimeter accu- 
racy. Libraries can cultivate the resulting high-resolution 3D data to 
provide a variety of modes for exploring, researching, preserving, or 
reconstructing physical historical features. The University of Virginia 
Library uses methods that implement the full scope of 3D data cura- 
tion through the collection, processing, archiving, and distribution 
of data and its derivatives to the scholarly community. The applica- 
tion of these methods as an intrinsic role within libraries opens up 

a potential area of information science that can be identified as 3D 
cultural heritage informatics (LIB3DCHI). More specifically, LIB3D- 
CHI can be described as a set of techniques that yield primary source 
data derived from the existing conditions of historic and culturally 
relevant objects, places, and sites. 3D technologies such as Web3D, 
computer-aided design (CAD), 3D printing, and virtual reality can 
help make a stronger connection to these objects, places, and sites by 
providing access to measured data about them through sensory-im- 
mersive technologies. Libraries are promoting democratized access 
to 3D assets as a means of providing new forms of knowledge to the 
scholarly community. 
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Introduction 


o maintain the connection between the physical elements of 

history and their historic narratives, it is essential to docu- 

ment the features of artifacts, architecture, and historic sites, 
both active and derelict. Data collection technologies such as laser 
scanning, photogrammetry, and other 3D data recording techniques 
and devices allow the documentation of the dimensions, scale, and 
geometry of a historic place or object with submillimeter accuracy. 
Libraries can curate the resulting high-resolution 3D data and 
provide a variety of modes for exploring, researching, preserving, 
or reconstructing physical historical features. The application of 
these methods within libraries opens up a potential area of Library 
Information Science that can be identified as 3D cultural heritage 
informatics (LIB3DCHI). At the UVA Library, LIB3DCHI is defined 
through the four fundamental stages of collecting, processing, ar- 
chiving, and accessing 3D data. 


Collecting 3D Data 


At the heart of LIB3DCHI is the acquisition of precision data. Docu- 
mentation of the existing physical conditions of material cultural 
heritage provides the fundamental base layer of data for LIB3DCHI. 
Producing reliable 3D data requires precision measuring instruments 
and techniques. 3D documentation technologies such as laser scan- 
ning, structured light scanning, and photogrammetry can provide 
measured data with millimeter to submillimeter precision. These 
technologies can provide two types of fundamental information 
about an object: surface geometry and color texture. Laser scanners 
and structured light scanners are metrology devices, and their main 
function is to collect surface data in three dimensions during the data 
collection phase. Lasers can detect surface variations from 2 to 0.2 
mm, depending on the sensor. External measuring devices such as a 
meter stick or calibrated scale bars should be included when photo- 
graphing the subject so measurements can be taken from the images 
and applied to the data when it is processed using photogrammetry 
software such as Agisoft Photoscan Pro (measurement is not allowed 
in Photoscan Standard). 

Choosing the most appropriate technology to yield the most ef- 
fective data depends on the subject matter to be documented. Scale 
and surface characteristics are the two most important features in 
determining which tool to use. For laser-based documentation tech- 
nologies, shiny surfaces, mostly black, or mostly white, semitrans- 
parent textured surfaces may generate enough distortion to corrupt 
the surface data or even make it impossible to collect the data at all. 
Because of the properties of light, lasers tend to reflect off shiny or 
white surfaces, while they are absorbed into black textured surfaces. 
Structured light and photogrammetry may work better for such sur- 
faces, as these techniques use less concentrated light to gather infor- 
mation. Structured light flashes a light pattern onto a surface, while 
photogrammetry uses images of the surface to generate data based 
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Fig. 2-1. Challenging surfaces: 
semitransparent, white textured 
quartz arrowhead (right image 
courtesy of Ben Ford, Rivanna 
Archaeological Services LLC) 


Fig. 2-2. Stages of 3D data 
processing from point cloud to mesh 


on triangulation from a camera pair. As long as the surface is kept 
consistently lit with a minimum of texture changes like shadowing, 
reflections, or color shifts, then either of these techniques may suc- 
ceed where laser light may fall short. The quartz arrowhead in figure 
2-1 shows conditions that are not conducive to the use of laser scan- 
ning, but are more amenable to structured light scanning. 

3D data collection techniques yield raw data in point cloud 
form. For example, laser scanners function by shooting a beam of 
light into the space all around the device, recording millions of 
points from surfaces that intersect with the laser. Photogrammetry 
produces similar data by generating sparse and dense point clouds 
during processing. The raw point cloud data are commonly cap- 
tured in a proprietary file format, along with other metadata that 
may have been recorded by the scanning device. Raw data may not 
be readable in software other than what the manufacturer of the 
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device has provided. In this case, the data must be processed and 
exported to other formats such as PLY, OBJ, DAE, PTS, or X3D to 

be usable in other software. Point cloud data can be converted to 
derivative 3D mesh formats and imported into other 3D software to 
be transformed into 3D content for a multitude of purposes. Figure 
2-2 shows scan data from Gutzon Borglum’s The Aviator at different 
stages of data processing from point cloud to refined mesh, which is 
the most commonly used form of 3D content. The processing phase 
must preserve as much precision as possible of the raw point cloud 
data as they are converted to a 3D mesh. 


Processing 3D Data 


In documenting a site, place, or object, it may be necessary to collect 
several datasets to cover the entire surface of the subject. A scan- 
ning device can capture only surfaces that are visible to the scanner. 
Thus, in most cases, either the scanner or the subject will need to be 
repositioned to ensure that all surface data have been captured and 
the subject has been fully documented. Multistage scanning sessions 
will yield separate datasets that must be registered together to pro- 
duce one cohesive dataset that accurately represents the documented 
subject. 

3D data registration techniques depend on the technique used 
for documentation (e.g., laser scanning, photogrammetry). Targets, 
and other registration devices, such as spheres, checkerboards, and 
coded targets, can be placed in the environment during the scanning 
process to help ensure that datasets are merged accurately. Process- 
ing software will recognize standard targets and use them to precise- 
ly fit datasets together. Current software is able to find similarities 
between datasets, called correspondences, to perform registration 
without targets. 

It must be emphasized that data are at the heart of LIB3DCHI. 
3D data are commonly misunderstood as “3D models” or “3D 
pictures” in discussions of the content produced through 3D scan- 
ning. However, 3D documentation technologies provide the actual, 
measured conditions of a place or thing, not just an appearance of 
or similarity to the subject. Object dimensions can be obtained from 
measurements between any two points in a point cloud, and point 
clouds can contain millions to billions of points, yielding a multitude 
of dimensional measurements. 

The fundamental structure of 3D point cloud data can be re- 
duced to values for X,Y,Z,i,R,G,B such as the contents of the PTS file 
seen in figure 2-3. XYZ describes a point existing along three axes in 
space. RGB is a color value attributed to that point in space. The vari- 
able, i, is an angle of incidence value that describes how light is inter- 
acting with the surface at that point in space (California Department 
of Transportation 2018). 

Along with the fundamental data, raw data from a scanner may 
include metadata that are produced by other sensors in the measur- 
ing device, including a global positioning system (GPS), altimeter, 
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Fig. 2-3. ASCII text readout of PTS 
formatted file contents showing 
X,Y,Z,1,R,G,B 


temperature, resolution settings, and other information specific to 
the conditions in which the data were collected. The data can be out- 
put to a variety of modes that include, but are not exclusive to, 3D 
CAD-type modes, such as files for 3D printing, VR, Web3D, or inter- 
active 3D environments. It is important for the archival record of the 
data to document the conditions under which the data were collected 
and produced. Careful archiving of source data and documentation 
metadata not only preserves measured 3D data, but also ensures re- 
peatability of data collection and processing methods, facilitating the 
reproducibility of data output (Pedersini, Sarti, and Tubaro 2000). 


Archiving 3D Data 


Libraries must develop effective means to balance preservation and 
access in archiving the diverse forms of 3D cultural heritage data. 
Archiving strategies can be placed on a spectrum of preservation and 
access from long-term storage to immediate use. At one end of the 
spectrum are dark archives provided by the Academic Preservation 
Trust (APTrust), a collaborative consortium of higher education insti- 
tutions managed by UVA; the APTrust’s goal is to develop strategies 
for storage and successful retrieval of all forms of data going into the 
future.’ APTrust provides storage of data in perpetuity and is not de- 
signed to allow immediate retrieval. 

On the other end of the spectrum are archival solutions that offer 
immediate, open access to data. Developed at Harvard University, 
Dataverse is an open source, open data platform that has the ability 
to ingest just about any data file format and then publish it openly to 
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Fig. 2-4. Dataverse interface 


the world.* The UVA Library favors an open access approach to ar- 
chives, incorporating Dataverse into its scholarly institutional reposi- 
tory, Libra, to allow open access to scholarship created by the UVA 
community. 3D formats can pose a variety of challenges to most re- 
positories because of their diversity in format types, but Dataverse is 
extremely flexible. Once data have been uploaded and published in 
the UVA Dataverse, a SOLR script creates searchable facets in Virgo, 
the UVA Library’s search and discovery platform. Search terms in 
Virgo depend primarily on the parsing of key terms and description 
fields in Dataverse. 

Dataverse allows open access to students, faculty, staff, and 
other teachers and researchers who wish to discover and download 
3D primary source data, and then incorporate 3D data into their re- 
search and pedagogy. Although 3D technologies are widely utilized, 
little is understood of the differences in 3D formats. Understanding 
3D is dependent on fully understanding how the content is gener- 
ated. 3D content can be manually modeled using authoring tools 
like Sketchup, 3D Studio Max, or Blender. 3D data can be generated 
using laser scanners or through photogrammetric processes. Source 
content is different from the derivatives that are exported from 3D 
authoring, editing, and optimization software. Source content may 
be usable only by the tools that created it or the software that was 
used during data acquisition. Content must be exported to formats 
that are contingent on context, such as 3D printing, CAD, and VR. 
Problematically, Dataverse will ingest a variety of source files and 
their derivatives and dependencies, and present the content as a list 
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THE FRALIN MUSEUM OF ART 


Fig. 2-5. Drupal interface with 
3DHOP viewer for accessing 3D data 
in UVA Library Virgo 


AT THE UNIVERSITY OF VIRGINIA 


Corinthian Kotyle 
Greek Vases Collection 
Collect, Care, Conserve, Curate: The Life of the Art Object 


™ Greek Kotyle, ca. 600-575 BC 
Corinth, Middle Corinthian black-figure, animal style, 
Archaic 
Ceramic, 3 1/8x 57/8 x 4 1/4 in, 7.94 x 14.92x 10.8 
cm 
Gift of Joanne B. Robinson, 2008.12 


Authoritative Record(s) 


View this object in the Fralin collection 


Corinthian Kotyle 
3D model of the Greek Kotyle 


Corinthian Kotyle 
3D model of Corinthian Kotyle in STL format 


Corinthian Kotyle 
Photograph of the Greek Kotyle, ca. 600-575 BC 


with no visual reference (figure 2-4). Therefore, scholars unfamiliar 
with 3D formats may have difficulty understanding how to use a 
particular file that is available in Dataverse. 

In collaboration with the UVA Library, the UVA Institute for 
Advanced Technology in the Humanities ([ATH) is developing a 
more user-friendly interface to access Dataverse assets and serve 
as an intermediary between Dataverse and Virgo. The interface 
uses the open-source web 3D viewer 3D Heritage Online Presenter 
(3DHOP) to provide an interactive 3D model for users to explore 
the data before download. It provides links for individual content 
formats according to potential use, for example, PTS, XYZ, or E57 for 
point cloud data or OBJ, PLY, STL, X3D, or DAE formats for mesh 
data. The interface in figure 2-5 presents the subject graphically and 
contextually with links to the repository, museum, or other collec- 
tion source where the original subject is kept, providing a bridge 
between the technical data, or provenance, and contextual metadata 
(D’ Andrea and Fernie 2013, 138-139; Denard 2009, 13). 
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Accessing 3D Data 


Once LIB3DCHI data have been archived and made openly acces- 
sible, scholars can use the data for a variety of academic purposes. 
Libraries are employing 3D content experts to help the scholarly con- 
stituency with a variety of tools, techniques, and information to use 
3D data most effectively. The following examples illustrate 3D data 
use at the UVA Library. 


3D Data Collections 

The 3D Greek Vase Scanning and Printing Project (3DGV) project is 
an ongoing collaborative effort between UVA archaeology professor 
Tyler Jo Smith and the UVA Library. In early 2015, Smith received a 
grant to study 3D documentation and 3D printing of artifacts from 
the Fralin Museum of Art at UVA.‘ Vases and other, similar artifacts 
of the Classical Greek period were scanned using a Creaform Zscan- 
ner 700CX laser scanner at a precision of 0.35 to 0.25 mm to collect 
the 3D surface and color texture data of the original. The data were 
processed and refined for 3D printing at a 1:1 scale in ABS and PLA 
plastic, typical to most printers today. While these types of printers 
do not replicate color texture from the 3D data, the geometric surface 
data can yield a nearly exact replica of the original vase’s form. This 
enables students engaged in archaeology, art and architectural his- 
tory, and other fields to directly handle artifacts, reducing stress and 
potential degradation of the original. This project continues today as 
new students are exposed to 3D documentation processes. 

The dataset from the documentation of Fralin Greek artifacts has 
become a collection on the UVA Library’s Dataverse, and the schol- 
arly public can access all of the data openly through Virgo. Museums 
such as the Fralin are keen to open collections data to the public with 
the understanding that a Creative Commons CC0 license enables the 
wider community to download and gain full access to the data, un- 
fettered by restrictive copyright protections. The Fralin has engaged 
in other projects with the UVA Library to document and provide 
access to other cultural heritage artifacts. In the spring of 2017, the 
Fralin hosted an exhibition entitled Collect, Care, Conserve, Curate: The 
Life of the Art Object, which explored various techniques and tools 
that museum conservators use for collection stewardship. 

Figure 2-6 shows artifacts from the Fralin Mesoamerican and 
African collections that were 3D scanned and printed to provide fac- 
similes of artifacts for museum visitors to pick up and explore while 
the originals remained safe, but visible, behind glass. In addition to 
the physical reproduction, a 3DHOP web viewer was made avail- 
able on an iPad kiosk for visitors to explore the fully color-textured 
object in 3D. Providing multiple forms of interaction from multiple 
derivatives of the 3D documentation data can bring scholars and the 
curious public alike closer to the physical reality of tangible cultural 
heritage. 


*http:/ /archaeology.virginia.edu /3d-greek-vases.html 
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Fig. 2-6. Collect, Care, Conserve, 
Curate: The Life of the Art Object 
exhibit 


3D Scan-to-Print Technology for Accessibility 

3D scan-to-print technology can be an effective solution for connect- 
ing the general public with invaluable artifacts and historic objects, 
but it can also be used to enhance interaction and appreciation of 
artwork for people with disabilities. At the Gari Melchers Home and 
Studio Museum at Belmont House, University of Mary Washington, 
Michelle Crow-Dolby teaches young children about the artwork of 
early twentieth-century artist Gari Melchers. She often works with 
children who have low vision disabilities that prevent them from 
fully participating in classroom activities. She looked to the UVA 
Library for help in finding a technical solution for these students. 
Crow-Dolby recognized the work that the UVA Library had done 
with 3D scanning combined with 3D printing and saw an opportuni- 
ty for productive collaboration. The Library once again used a Crea- 
form Zscanner to 3D-scan sculpture from Gari Melchers and 3D print 
a 1:1 replication of the original. Crow-Dolby now uses this technique 
to allow the children to interact directly with the sculpture, which 
visitors to the museum could not normally handle. 

By providing a replica of an original artifact or artwork based 
on precision measured data, 3D scan-to-print output could be con- 
sidered a form of 3D data in its own right. Digital data gathered by 
3D documentation techniques are encoded in the physical form of 
the 3D print. It must be noted, however, that the physical replication 
of an artifact is not exact. The 3D scanning and printing processes 
degrade the precision of the information derived from the original 
subject. At best, laser scanning can provide a precision of only 0.25 
mm of the surface of the original. The many minute details and nu- 
ances that exist below 0.25 mm are not recorded. Additional informa- 
tion, such as high-resolution photographs, scholarly accounts, and 
more descriptive metadata fields, must accompany the data record 
for preservation. 
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Fig. 2-7. THLib Lhasa VR project 
interface in ESRI CityEngine Web Viewer 


5 http: / / www.thlib.org / 


Historic 3D Reconstruction: LhasaVR 

Before the development of 3D acquisition methods, manual 3D 
modeling was a standard method for 3D documentation of archi- 
tecture and artifacts. Just as plan, section, and elevation renderings 
will remain the standard for architectural documentation, historic 
3D reconstruction continues to be an effective means for generating 
cultural heritage data. This is especially true of non-extant places and 
things or time-dependent, ephemeral conditions of architectural and 
archaeological features that may have undergone renovation, res- 
toration, or reconstruction. With careful research into historic docu- 
mentation of a place or thing, 3D modeling can digitally re-create a 
cultural feature. 

A massive 3D reconstruction project at UVA is the Lhasa VR 
project with the Tibetan and Himalayan Library (THLib).> The main 
effort of this project was to build a comprehensive historical GIS of 
the Tibetan capital of Lhasa prior to the 1959 occupation by Chinese 
forces. More than 3,000 buildings have been razed or significantly 
altered since then. With the use of early twentieth-century maps and 
early high-altitude photographs, a comprehensive 2D map was cre- 
ated of the city and its immediate surroundings in the Lhasa-Kyichu 
River Valley. More than 3,500 features were identified by using Esri 
ArcMap mapping tools. Each feature was linked to the THLib data- 
base to create a map enriched by information. The 2D map was then 
converted to a 3D model by using the ESRI CityEngine procedural 
modeling tool and data from the THLib database to auto-generate 
buildings according to their historic characteristics. Higher resolution 
3D models were produced in Autodesk 3D Studio Max and then im- 
ported into the CityEngine 3D GIS model. 

The view of the 3D model in figure 2-7 is from the CityEn- 
gine Web Viewer utility, a free Web3D viewer for 3D GIS models 
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created with CityEngine. 3D models created in 3DS Max were 
based on architectural information provided by the nonprofit, non- 
governmental organization Tibet Heritage Fund. Projects such as 
Lhasa VR expand the use of 3D reconstruction models by using 
information systems such as GIS or BIM to link historic informa- 
tion to 3D data. 


VR Pedagogy: Virtual Museums 

The transformation of 3D data into pedagogical content is the fun- 
damental goal of the 3D cultural heritage informatics process. The 
effective use of 3D content in the classroom is contingent on having 
tools that promote an engaging and meaningful experience. Immer- 
sive technologies such as VR can bridge the gap between physical 
history and cultural heritage data. 3D technologies play a crucial 
role in providing alternative modes of access to the physical world. 
Publicly accessible VR systems give users of both university and 
public libraries an opportunity to experience immersive content. The 
UVA Library provides public VR spaces that its scholarly community 
can use for teaching, learning, and playing. Playtime encourages 
students and faculty to become familiar with 3D content interaction 
and opens up a variety of potential tools and methods for interact- 
ing with 3D data. Using free software such as authoring tools in 

the Unity 3D game engine and the Steam VR platform, 3D and VR 
specialists at UVA teach pedagogical methods that are easy to learn 
and apply to a variety of academic fields. A method developed by 
Arin Bennett, VR/ AR specialist with the UVA Library Scholars’ Lab, 
and Will Rourk uses Unity 3D as the main platform to help students 
curate spatial narratives based on curriculum research topics. Nar- 
rative spaces are designed on the priniciples of museum spatial 
layout. A research topic is represented spatially by creating “rooms” 
in a virtual museum that relate to the arguments in a paper. The de- 
tails of the argument are expressed by images, text, audio, or video 
objects placed in a room much like objects in a museum exhibition. 
The rooms are usually designed for a linear experience much in the 
same manner that a paper would be read. Interactions with these 
spaces are usually kept simple, enabling, at a minimum, the ability 
to “walk” from one space to another to see images or read embedded 
or linked text. Students who want to add more interactive features or 
create more specifically curated experiential elements can do so by 
learning Javascript or C# scripting, which are both native to Unity 
3D. Scripting is not taught in the class, but learning how to build ad- 
vanced interaction by downloading premade scripts is encouraged. 
The class is designed for base level novices with little experience in 
3D modeling or content creation. 

Students then lead their classmates in a tour of their research 
narrative while providing their own scholarly narrative. The 3D con- 
tent is either output to a Web3D viewer in a webpage or converted 
into a virtual environment for an immersive exploration with VR 
headsets such as the HTC Vive or Oculus Rift. Both modes are net- 
worked and allow for multiple participation so that other students or 
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Fig. 2-8. 3DCHIVR meeting in VR at 
the UVA Academical Village (ca. 1820s 
CE) and Warm Springs Bathhouses (ca. 
late eighteenth century CE) 


faculty can join in from other parts of the university, as well as from 
external institutions. 


Open Access VR: 3DCHIVR 

In collaboration with a team of researchers from James Madison Uni- 
versity, the University of Mary Washington, and James Madison’s 
historic home at Montpelier, staff at the UVA Library are researching 
methods for enabling open access to LIB3DCHI content in VR sys- 
tems. Each of these institutions engage in VR systems-based interac- 
tion with their own constituency, while acknowledging that this con- 
tent can easily be shared across networks with the broader scholarly 
community. This multi-institutional team has entitled their effort the 
3D Cultural Heritage Informatics Virtual Reality (3DCHIVR) project, 
and their goal is to develop a platform for sharing content that is for- 
matted for use in VR systems. The system is to be based on the mod- 
el of open access, search, and discovery with Dataverse and Virgo at 
the UVA Library. To model how this system would work, the team is 
using the Steam VR platform to provide LIB3DCHI content collected 
and processed from laser scanners, photogrammetry, and historical 
3D reconstruction. Participants sign up for free with Steam VR and 
can connect to one another in VR for a multiparticipant exploration 
of 3D documentation data of historic places. 

Figure 2-8 shows a meeting taking place in data produced from 
two sites. At left is a 3D model of the Anatomical Theater, Academi- 
cal Village, University of Virginia, produced for the JUEL Project (Jef- 
ferson’s University, the Early Life) by IATH. At right is 3D data from 
laser scanning and aerial photogrammetry performed at the Warm 
Springs Bathhouses (ca. late eighteenth century) in Bath County, 
Virginia. 3D documentation data were converted into 3D VR content 
with little to no editing so that participants can explore an authentic 
replication of an actual historic site. Through networked VR systems, 
historical data can be easily distributed to the wider scholarly com- 
munity for a firsthand experience of cultural heritage features. An 
open source, open access solution is the desired outcome. 
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Conclusion 


The methods of LIB3DCHI help bridge the experiential gap between 
subject and scholar. One of the goals of scholarly research is to be- 
come more closely acquainted with cultural heritage through its 
physical record. Access to collections traditionally means examining 
artifacts behind glass, visiting a historic building in a remote or dis- 
tant location, or working with media that express a likeness of the 
original such as in photographs, video, audio, or conjectural physical 
or digital 3D modeling. In most cases, the researcher is physically 
separated from the subject and connected primarily through media 
or other facsimile representations. 3D documentation technologies 
and methods such as laser scanning, photogrammetry, and histori- 
cal reconstruction modeling can provide precise data that are useful 
for creating accurately measured representations of artifacts, archi- 
tecture, archaeological sites, and other elements of physical cultural 
heritage. Web3D technologies can help conveniently distribute 3D 
content and data through web browser interfaces. VR is a powerful 
mode for exploring cultural heritage 3D content, as it immerses the 
scholar in a life-sized, scaled representation of historic subjects with 
tools to explore, annotate, and make inferences while sustaining dis- 
course with other scholars and researchers within the global commu- 
nity. 3D printing can put accurately reproduced facsimiles directly 
into the hands of students and researchers who may otherwise have 
only limited tactile, visible, or other sensory interactions with the 
original subject. 

3D technologies can strengthen a user’s connection to the subject 
by providing access to measured data of historic objects, places, and 
sites through sensory-immersive technologies. Libraries are foster- 
ing democratized access to 3D assets as a means of providing new 
forms of knowledge to the scholarly community. By considering the 
creation, access, and preservation of 3D data, LIB3DCHI techniques 
help to ensure that 3D assets are meaningful and useful for research 
and pedagogy long into the future. 
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Chapter 3 


Virtual Reality for Preservation: 


Production of Virtual Reality Heritage Spaces 
in the Classrooom 


Zebulun M. Wood, Albert William, and Andrea Copeland 


Abstract 


The Bethel AME Church was the oldest African American church 

in Indianapolis. In November 2016, the congregation moved out of 
downtown, and the building that had housed the congregation since 
1869 was sold. It is now being redeveloped into a hotel. Through the 
Virtual Bethel project, faculty and students in the Media Arts and Sci- 
ence (MAS) program at Indiana University—Purdue University India- 
napolis (IUPUI) created a 3D virtual space of the physical sanctuary 
to preserve the cultural heritage of Bethel. During its creation, Vir- 
tual Bethel served as a curricular and co-curricular experience for the 
undergraduate students in the 3D graphics and animation specializa- 
tion within class N441 3D Team Production, which was co-taught by 
Albert William and Zebulun Wood. Virtual Bethel, finished in 2018, 
was the first historical and cultural preservation project that used VR 
within our class, program, school, and Indiana University (IU) cam- 
pus. Users can interact with various types of primary sources (e.g., 
photographs, video, audio, text) to learn about the underrepresented 
history of African Americans associated with the church. Virtual 
Bethel was created in a series of classes within the MAS Program in 
the School of Informatics and Computing (SoIC), IUPUI. Methods of 


Virtual Bethel was funded by an Indiana University New Frontiers Grant in 2017. The authors would like to thank the 

team of the Advanced Visualization Lab (AVL), IUPUI: Thanks to Jeff Rodgers, Chauncey Frend, Tyler Jackson, and Michael 
Boyles for their unrelenting support of our programs and all-too-often thankless hours they put into our projects by fielding 
questions and inspiring all. We thank the students of Virtual Bethel, first Luke Brown for coming onto the project to embed 
and build a VR storytelling database, the first of its kind at IU, and for being at every public showcase and documenting 
everything. Thanks to original team Tyler Jackson, Rachel Davidson, Bryan Dinkens, Thomas Springer, Roxanne Wheeler, and 
Charles Yu, and to Skip Comer and Lisha Chen for creating our website and the ability to provide Virtual Bethel to the world. 


We thank Online Resources, Inc. for the original 3D scan of Bethel AMC, and Kisha Tandy for providing research and for 
assistance in curating the Virtual Bethel Vignettes. Finally, we thank the Bethel AMC Congregation, especially Olivia McGee- 
Lockhart, for curating the mission, public showcases, vignettes, and accuracy of content within Virtual Bethel. Thank you for 
your continued partnership with our students, school, and university. 
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teaching a team of students to preserve historic spaces using VR are 
discussed, as are our philosophies toward productions when work- 
ing with varying stakeholders’ priorities related to data preservation, 
asset preservation, and cultural preservation. 


Project Background and Significance 


he Media Arts and Science (MAS) program in the School of 

Informatics and Computing (SoIC) provides undergraduate 

courses in 3D production and visualization. As a project- 
based course, N441 3D Team Production is designed to involve 
community partners in providing students with real-world, context- 
driven, hands-on learning opportunities. The course focuses on the 
creation of high-end, broadcast-quality animations through team- 
based learning. Students learn skills in areas related to production 
in a 3D project. These skills include preproduction tasks such as the 
development of a story, script writing, research, conceptual drawing, 
storyboarding, animatics, and project management. Production skills 
are explored in 3D asset creation, time management, file manage- 
ment, sound, and title sequences. Postproduction processes include 
final rendering, movie creation, and formatting for various playback 
devices. More recently, the program has embraced projects imple- 
menting and leveraging emerging technologies such as 3D printing, 
VR, and augmented reality (AR). 

Founded in 1836, the Bethel AME Church was once a vital part 
of a thriving African American community in the heart of the Indi- 
ana Avenue Jazz District. Before that, Bethel played a vital role in the 
Underground Railroad. Bethel has significant meaning not just in Af- 
rican American history, but also in the local heritage of Indianapolis. 
Recently, the church site was rezoned for redevelopment, leaving the 
historic building—and the materials housed in the church archive for 
more than 162 years—in a vulnerable position. 

Although the effort to digitally preserve the at-risk physical 
space is not innovative in and of itself within cultural preservation 
domains (e.g., Arc/k Project,! CyArk,? Iconem,? MasterWorksVR’), 
the Virtual Bethel® project incorporates associated digitized and 
born-digital archival materials into the virtual space to provide a 
new way of learning history and interacting with Bethel’s primary 
sources. The project intends to develop this space as a virtual learn- 
ing environment for undergraduate students’ history and primary 
source education. The methods used to develop Virtual Bethel by 
engaging undergraduate students will be relevant to studies of other 
historic sites and archives with similar ambitions. 


http: / /arck-project.org 

2 https: / / www.cyark.org 

3 http:/ /iconem.com/en/ 

* http:/ /masterworksvr.com 

5 https: / / comet.soic.iupui.edu/bethel / 
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Community: Respecting the Heritage and 
Institution’s Members 


Communication among the undergraduate N441 3D Team Produc- 
tion class, Bethel’s church membership, and local historians connect- 
ed the students easily with Andrea Copeland, chair of the Library 
and Information Science Department in the SoIC at IUPUI. Copeland 
focuses her research on facilitating connections between groups typi- 
cally underrepresented by heritage institutions and community pres- 
ervation infrastructures. Her research finds that trusted relationships 
are essential for reducing social distances and building connections 
among individuals, institutions, and knowledge. She has worked for 
several years with congregants from the Bethel AME Church. Work- 
ing together on this project, we learned valuable lessons that can be 
used to support future community-driven heritage projects. 


Course Description and Necessary Skills for 
Restoring 3D Scanned Structures 


The N441 3D Team Production class allows students to work as a 
group and emulates the collaborative team environment found in 
the media and animation industry. The goal of the course is to bring 
students together to work on a common project. At this point in their 
undergraduate degree program, students have completed a number 
of prerequisites, including intermediate courses in 3D modeling, tex- 
turing and lighting, and animation. Virtual Bethel served as the first 
cultural preservation project using VR as a medium to engage stu- 
dents with the aim of educating the public. Regardless of the project 
assigned, students are encouraged to bring their existing knowledge 
and specialty (e.g., modeling, unwrapping, shading, materials, light- 
ing, game development) to the team and investigate new skill sets 
that they may want to develop or that are needed to facilitate the 
success of the production. 

We have found through teaching this course for five semesters 
over five years that students engage in this class for a variety of rea- 
sons with a range of positive outcomes. The course allows them to 
work together on a single project synergistically. Students can apply 
their existing skills and also find ways to implement other interests 
that they may not have yet developed. The course teaches them team 
dynamics and develops skills essential to the success of group work, 
including communication, leadership, organization, and account- 
ability. Past class assessments indicate that the experiences students 
have in this class are far beyond regular class work in contributing to 
a larger team and serving the community, that they relish this experi- 
ence, and that the overall result is a very satisfying academic exercise 
(Lombardi 2008; McLeod 2017). 

At the beginning of class, we assess student strengths to com- 
pare the existing skill sets with a project’s goals. We carefully look at 
student abilities, often recruiting students that we feel might benefit 
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from the experience and who can contribute specifically to certain 
portions of the project as it develops. We expect strong student lead- 
ership from within the group and encourage students to be account- 
able to their peers rather than to the instructors. We set up a data and 
communication structure that ensures all students begin interacting 
as soon as the class starts. We encourage the students to adopt a 
communication system that is easiest for them; many times, students 
have opted to use Facebook or other social media platforms. They 
also communicate daily and share files through a system used in the 
Indiana University infrastructure called IU Box. 

Technical knowledge of the craft (in this case, animation) is, of 
course, critical to the success of the project; yet the students are en- 
couraged to research the topics on their own early in the semester. 
Supplementary materials are gathered and provided in our course 
materials on Canvas, the campus learning management system. 
Knowing the history of the subject or the background of the story 
gives students more interest in the success of the project and their 
role within it. 


The Classroom as a 3D Production and 
Preservation Studio 


In the fall semester of 2016, our N441 class was presented with the 
opportunity to re-create the historic Bethel AME Church in India- 
napolis. We felt that a VR experience using 3D models and textures 
would be a powerful method of telling the story of this church. We 
also felt that this opportunity for community engagement was a per- 
fect fit with the collaborative nature of the class, as IU strives to high- 
light its engagement and impact throughout the state. We presented 
the students with this opportunity in the first class meeting. They 
considered this a great use of their skills and time, and they were 
excited to be working to give back to the downtown Indianapolis 
community. 

During our second class meeting, all students and faculty visited 
the church, which was within walking distance of our classroom, 
to experience the space that we would re-create. In early September 
2016, on a warm, late afternoon with sunlight streaming through 
the stained glass windows, we spent more than two hours exploring 
every corner of the church. We took about 2,000 high-quality photo- 
graphs to use as reference for the 3D models and textures that would 
go into the VR experience. We used a GigaPan robotic camera system 
to capture images that were stitched together to form highly de- 
tailed panoramas of the interior, and a Ricoh Theta camera captured 
low-resolution 360-degree images. All of these photographs were 
cataloged, archived, and used to help re-create the digital space. 
Measurements of some structures were also taken as references for 
scaling the models. 

Before the start of the semester, Online Resources, Inc., a lo- 
cal 3D scanning company, had created laser scans of the interior of 
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the Bethel AME Church for the project with a 100mm Surphaser 
LiDAR laser scanner. This data set was approximately 2.9 GB and 
contained 39 million polygons. Tests of the scanned data using 

the Unreal 4 game engine outputting to an HTC Vive VR system 
showed that, because of the density of the scanned mesh and size 
of the data set, we had to greatly reduce the scan so we could view 
Bethel in VR. The group decided early on to use the scan as a type 


Fig. 3-1. The original LIDAR scanned 
data (top), retopologized interior 
(middle), and fully textured and lit Virtual 


of digital tracing paper, an excellent reference resource that could 
speed up our modeling workflow (figure 3-1). One of the first steps 


Bethel ready for viewing in VR (bottom) taken as we started to develop the church interior was to bring the 


ne 


Laser scan model provided i Online Resources, Inc. 


Fully textured and lit Virtual Bethel 


reduced scanned data into Unreal and 
scale it correctly. We simply placed a 
six-foot box representing the size of a 
human and compared it with the ob- 
jects and space within the scene. 

Students used Autodesk Maya 2016 
as the software to model all of the as- 
sets. The scanned data were reduced 
using Pixologic Zbrush’s decimation 
tools, then brought into Maya and 
scaled to their proper dimensions for 
use as a template. Objects that had been 
measured by students were used as 
references, and then the scanned model 
was brought to its proper relative scale 
based on unit settings (mm, cm, meter) 
in Maya. 

Using the scanned data and refer- 
ence photographs, our modelers were 
able to determine the basic shapes of 
each part of the specific objects and be- 
gin manual reconstruction. We needed 
to be cognizant of the number of poly- 
gons that this model contained so that 
it would show enough detail, but the 
number could not be so high as to hin- 
der the Unreal 4 engine executable file 
when it was brought to VR and dis- 
played in stereoscopy. 

Production meetings, in which 
iterations of the virtual environment, 
individual assets, and interactions 
in the virtual environment were dis- 
cussed, were held at the beginning of 
each week’s class in a conference room, 
instead of our usual classroom. This 
allowed students to focus on all of the 
assets, brought them together as a team, 
and showed them how their contribu- 
tions were affecting the big picture. The 
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team reviewed each student’s work weekly, addressed concerns 

as needed, then tasked the students with moving forward on new 
assignments. We, the instructors, stressed to each student that oth- 
ers relied on their progress and thus adherence to deadlines was 
mandatory. Production meetings were followed by visits to the IU 
Advanced Visualization Lab® to view the progress on the VR envi- 
ronment within the Unreal game engine. After the team meetings, 
the rest of the class was dedicated to lab/ production time for all 
members to spend time interacting, working, and receiving indi- 
vidualized instruction from both faculty and team leads as needed. 
On artistic or preservation considerations, the students considered 
feedback from the client/ public and adjusted the interactives based 
on the audience’s reaction to the environment, primary resources 
within VR, or even the VR hardware itself. As we neared comple- 
tion, some church congregation members and the church’s pastor 
visited to see our progress. It was surreal to watch the members 
find the pew where they had always sat and to observe the pas- 
tor stand in his favorite spot to give a sermon. The students saw 
the impact of their work on the visitors and started to understand 
how this project was important for the community. This was a very 
powerful visit for all involved.’ 

As students completed 3D models for populating the VR envi- 
ronment, they kept records of their progress on a spreadsheet. As 
digital models were completed, students began unwrapping the 3D 
objects so they could be textured using Allegorithmic’s Substance 
Painter. This process applied materials and colors to the 3D models 
to give them a sense of realism. To simplify the identification of 
particular materials, an internal team library was created for stu- 
dents to access materials that had been identified in Bethel. Materi- 
als represented surfaces in the real world (e.g., types of wood [cher- 
ry, oak, poplar], paint [glossy, matte, aged], metals [chrome, copper, 
iron], plastics [silicone, rubber, shiny, glossy]). As models were 
painted within Substance Painter, texture maps were exported for 
use in Unreal 4’s physically based rendering (PBR) shaders, which 
experts in the video game and architectural visualization industry 
commonly use to make surfaces appear realistic. 

After all models had PBR materials applied and were loaded 

into Unreal, various processes were used to optimize the scene. 
For example, the textures were tested to see whether there were 
errors, lighting was added to the scene to simulate light coming 
through the windows, navigation controls were optimized, and 
teleportation locations were created to permit navigation in the 
environment. 


6 https: / /kb.iu.edu/d/apel 

7 See a short video of Virtual Bethel in early production used to solicit additional 
support: Virtual Bethel Solicitation, October 14, 2016. Available at https: / / vimeo. 
com/ 187085145 
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Building a Campus Infrastructure for Virtual 
Preservation of Cultural Heritage 


In projects such as Virtual Bethel, the prior knowledge, resources, 
and communication ability of support staff, faculty, project partners, 
and client partners are critical to project planning. Readily avail- 

able hardware and software can ensure the success or guarantee the 
failure of projects. The MAS faculty have made several choices and 
adopted specific philosophies to ensure success across media projects 
of many kinds. Following are discussions of specific considerations 
concerning VR projects involving the digital preservation of spaces, 
including technical considerations; associated costs; student technical 
and artistic competencies; variations in student level of confidence 
and leadership; variations in peer-to-peer organization, communica- 
tion, and accountability; and student learning outcomes. 


Technical Considerations 


The MAS program faculty and students make every effort to stay 
software agnostic, especially in relation to game development en- 
gines. In an age when software updates are daily and software 
companies are purchased every hour, it is impossible to anticipate 
which updates, plug-ins, or software will stop being supported or 
will be changed entirely. We encourage our faculty and students to 
test, vet, and hone their skills on multiple platforms. The Unity game 
development engine tends to enable easier porting to various head- 
mounted devices (HMDs), mobile devices, and app stores, while Un- 
real, until recently, has supported only higher rendering and realism 
capabilities. We built Virtual Bethel using the Unreal game engine 
because we wanted to develop contained systems for porting Virtual 
Bethel onto full VR, mobile devices, and web environments for maxi- 
mum public access. This was a new process for us, and future proj- 
ects in the program will benefit from the lessons learned. 

Another important technical consideration is that 3D scans of 
objects, spaces, or both must be completed and delivered before the 
start of a production course semester. Geometry created from scan 
data is often best at the highest possible capture settings. Whether 
the data are captured via LiDAR, structured light, photogrammetry, 
or by other means, the bigger the dataset, the bigger the textures, and 
the larger and more frequent the photos, the better the end result. 
Given the temporal constraints of the academic semester, student 
productions cannot be delayed because of the need to recapture or 
find additional photography. 
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Associated Costs 


VR projects like Virtual Bethel have substantial costs. The total sup- 
port funded by Indiana University’s New Frontiers Grant® in 2017 
was considerable at roughly $59,000. Most of the funds covered the 
costs of student hourly labor outside of class time (texture artists, 
lighters, game developers, web developers, user experience [UX] 
researchers), faculty summer support, and a modest honorarium for 
our community partners. Nearly 10 percent of the budget was used 
to purchase server space for the website and invest in a mobile VR 
workstation with a laptop capable of showcasing iterations of Virtual 
Bethel to the public during its production so that students could re- 
ceive feedback (Copeland et al. 2018). 

Without an MAS or similar program, an institution is unlikely to 
have the necessary resources for such projects, including lab space, 
and hardware and software. The lab space, computers, display hard- 
ware, and 3D and game design software had already been purchased 
for teaching MAS students in the undergraduate and graduate pro- 
gram within SoIC, IUPUI. The core labs, IT 255 and IT 257, house 43 
computers with Cintiq displays with the latest software and hard- 
ware for film, game and VR art, production, and development, as 
well as VR hardware, respectively (figure 3-2). Unless this infrastruc- 
ture and the supporting IT staff already exist, projects like Virtual 
Bethel would be unreasonably expensive. 


Fig. 3-2. 3D (left) and VR production 
(right) labs at the School of Informatics 
and Computing 


In terms of labor, the three to five MAS students who worked 
during the digital replication of Virtual Bethel averaged a total of 15 
hours per week, followed by two students who averaged ten hours 
per week in labor in the second semester, and one student who 
worked over the last eight months embedding audio, interactions, 
and iterating on the story vignettes within Virtual Bethel once the 
project was funded at an average of 10 hours per week. Two stu- 
dents were paid outside of class after the first semester through the 
support of the IU New Frontiers grant for two additional semesters 


8 https: / / research.iu.edu / funding-proposals / funding / opportunities /new-frontiers / 
index.html 
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to continue the development of the interaction portions of Virtual 
Bethel; to embed a curation experience; and to refine standardized 
workflows for creating multidevice VR executables for any audi- 
ence or hardware such as Oculus, Vive (full VR), iOS and Android 
(mobile VR), and web-based versions of the experience (for those 
lacking HMD or mobile VR hardware). A graduate student with web 
development experience was also paid to develop the public facing 
website and store all of the content.’ 


Critical Decision-Making Points Through the 
Project Based on Stakeholder Feedback 


Several critical decision-making points arose through the project, 
particularly in terms of VR navigation, curation of Bethel’s story and 
history, implementation of audio and voice, protecting the cultural 
protocols of the Bethel congregation, and preservation community 
priorities with VR constraints. 


Virtual Reality Navigation 

During the first public showcase in the fall of 2017, we noted that 
many of the Bethel membership were elderly and took several min- 
utes to learn to use VR navigation. Some members also were unable 
to stand because of their health. The decision was made and imple- 
mented immediately after that session to provide two modes of VR 
navigation when showcasing Virtual Bethel with HMDs: one stand- 
ing mode with teleportation enabled, and a second sitting mode that 
would allow the user to transition to various locations preloaded in 
the Virtual Bethel sanctuary by clicking one button. Making the VR 
experience accessible to those who cannot stand or walk has become 
a key priority to building an alternative VR navigation interaction in 
projects for the future. 


Curation of Bethel’s Story and History 

After the space was initially showcased to the local membership, 
Bethel resident historian and Virtual Bethel Curator, Olivia McGhee- 
Lockhart and several heritage partners were given the opportunity 
to experience the Virtual Bethel space. They expressed gratitude at 
the re-creation of the space, but said that the experience felt flat and 
lacked a sense of story, import, or exploration. We knew the space 
alone was not enough to educate or leave an impact on an audience, 
so we decided to embed interactive story vignettes (figure 3-3). The 
vignettes included short written content and surrounded digital 
versions of content such as scanned newspapers, ledgers, and photo- 
graphs of Bethel and its membership. In the late fall of 2017, the team 
turned its attention to creating a database within Virtual Bethel so 
that it could outlive the VR team. Now Virtual Bethel can accept ad- 
ditional story vignettes that house various types of data. 


° https: / / comet.soic.iupui.edu/bethel / 
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Fig. 3-3. Examples of interactive 
story vignettes 
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Bethel AME church to grow in spreading 
‘the message of God and maintain similar 
donationsfor the year that comes and 
“your indebtedness will be paid and your 
church will be entirely free. Standby your 
church, your pastor, your officers and your 
God and all will be well." The 1895 annual 
report gives us insight in the social 
organizations that sprang from the Bethel 
Church. 


Implementation of Audio and Voice 

On Martin Luther King Day, 2018, we were showcasing the latest 
version of Virtual Bethel, which then contained 20 story vignettes; 
we noticed several audience members saying that they would love to 
“hear” ambient church members and McGee-Lockhart, as a critical 
part of the vignette experience. As a result, McGee-Lockhart record- 
ed audio for each vignette, discussing the history of specific parts of 
the church, and the audio was programmed to play upon interac- 
tion with the vignettes. This enriched the experience of the space for 
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users and enabled them to hear why each space was so important to 
its congregants and for the city of Indianapolis. 


Plan for Sharing 

Early in the relationship, because of the inexperience of project part- 
ners in both the game development processes and VR, we created 
contractual protections for Virtual Bethel that still make it legally 
difficult to share the project folders with interested parties. Our goal 
was to ensure safe, responsible, and ethical use of Virtual Bethel files, 
as we feared that ease of access could lead to unimagined and poten- 
tially harmful reuse of the digital files. We wanted to protect Bethel’s 
membership from the possibility that a game or other interactive ex- 
perience could be made based on the Virtual Bethel content without 
their express consent. 


Preservation Community Priority Versus VR Constraints 
Early in the project, stakeholders unfamiliar with the technical and 
hardware constraints of VR expected objects in virtual space to have 
the same level of realism and accuracy that objects have in real life. 
We explained to project stakeholders and community partners that 
varying levels of realism can be achieved based on the following fac- 
tors: student labor force size, ability, and available time. We are very 
proud of our students’ final version of Virtual Bethel and its level of 
realism /believability. In our opinion, it exceeds many other projects 
in both aesthetics and interaction. 


Looking Ahead 


As new technologies emerge, as virtual interactions become simpler 
to implement, and as alternate realities permit ever-richer experi- 
ences, new opportunities are continually emerging for research and 
application. Looking forward, we suggest some areas of research in 
which to invest prior to building a virtual recreation. 


Agnostic Platforms 

Project partners and stakeholders at IU used Virtual Bethel’s method- 
ology to recreate various spaces, but with significant differences; for 
example, some teams used Unity game development engine, while 
our team had used Unreal game development engines. This meant 
that project teams frequently could not use the virtual artifacts and 
interactions that others created. Developing across multiple software 
platforms can be cumbersome and often leaves one team feeling left 
behind the other. In MAS, we have committed to creating projects in 
both Unreal and Unity game development engines to both increase 
the employability of our students and to stay current with trends in 
these industry-leading applications for next-generation interactions. 
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UX Standards and Institutional Documentation 

There is little documentation for developing platform-specific inter- 
action standards for virtual reality. Documenting virtual interactions 
that do (and do not) work well, sharing the code that creates them, 
and explaining the reasoning and context behind the choices made 
during their creation is therefore critical to building a project man- 
agement and knowledge-sharing infrastructure. What works well in 
a full VR experience for navigation and interaction must be config- 
ured separately for mobile devices and again for web-based virtual 
experiences on a desktop computer. The Virtual Bethel team has 
committed to delivering VR environments for all major platforms 
and to documenting the processes of creating interactions for each 
platform to aid future IU teams. Creating tools that automate the de- 
velopment of navigation and interaction inputs across VR platforms 
and devices will expedite the testing, evaluation, and accessibil- 

ity of virtual reality-supported historical and cultural preservation 
projects. 


Advanced Capture Technologies 

Capturing 3D artifacts and spaces is becoming cheaper, more effi- 
cient, and more accurate every day. For example, we can now use a 
combination of photogrammetry-produced high-resolution textures 
with highly accurate spatial data from laser scanning—an approach 
that was not available to us when the project started two years ago. 
Documenting methods for combining geometry and textures from 
multiple imaging tools will be important as advanced capture tech- 
nologies and techniques change over time. 


Quantifying Authenticity 
Throughout the creation of Virtual Bethel, our core concern was to 
recreate the chapel for the congregants. Each of the stakeholders we 
spoke with—the engineers scanning the space, the librarians scan- 
ning documents and artifacts, our students, the Bethel membership, 
or the public at large—had their own definition of what was real, 
true, or believable in virtual reality. 
We began to frequently ask the following questions: 
¢ How can we quantify authenticity of virtual objects /spaces for 
different stakeholders when forced to remake spaces / artifacts for 
VR experiences? 
¢ What are the dependent variables in defining authenticity? 
¢ Do digital born artifacts have authenticity as a digital replica? 
¢ At what point does digitized content made for AR or VR not rep- 
resent the physical artifact on which it is based? 
¢ How much freedom does a 3D artist have? 


Much effort was spent educating all project partners and stake- 
holders on the geometry, texture, and lighting constraints of VR (and 
limitations of student ability) while also convincing them to capture 
the highest possible quality of scans for posterity. 

We believe that virtual reality will become the interaction 


Virtual Reality for Preservation: Production of Virtual Reality Heritage Spaces in the Classroom 


medium of choice for audiences to learn about and experience his- 
tory. Watching Bethel congregants, students, children and colleagues 
interact with, gain insights from, and understand the VR medium 
and how Bethel is being preserved in a new way leads our team to 
believe this is a much more natural, believable, and accessible me- 
dia environment with which to engage, educate, and entertain. As 
hardware adoption and the public’s comfort interacting with digital 
content increases, preservationists will no longer just preserve, but 
will also have the opportunity to lead the curation of and interaction 
with the objects, spaces, and time periods they protect (see Costa and 
Melotti 2012; Morcillo et al. 2017). 


Accessibility for All 

Media Arts and Science at IUPUI is committed to making available 
and representing its students’ and project partners’ hard work in as 
many ways as possible. Until full head-mounted displays are ubiq- 
uitous in homes around the world, we see it as necessary and ethical 
to create full, mobile, and web-based VR iterations of all of our proj- 
ects to ensure that all audiences can learn from the virtual spaces we 
create. 


Conclusion 


Throughout its creation, Virtual Bethel has benefited the faculty, stu- 
dents, librarians, preservationists, community partners, and, most 
important, Bethel church members. Positioning a 3D/ VR production 
as a focal point for heritage preservation inspired quick stakeholder 
buy-in, enthusiasm, and flexibility through collective understand- 
ing. All stakeholders embraced this emerging technology as a unique 
preservation method. The IUPUI Library has become a place to learn 
about emerging technologies, anticipate trends, and preserve the 
digital files of productions such as Virtual Bethel. The Virtual Bethel 
project has become an exemplar of what a library can offer its public 
and how an academic institution can leverage faculty and on-cam- 
pus resources while integrating its students into authentic and en- 
gaging curricular and co-curricular projects. Undergraduate students 
led the VR production day to day, and the result, with a bit of orga- 
nization and regular community feedback, was more than anyone 
could have imagined. We could not be happier. During its creation, 
Virtual Bethel inspired six other preservation projects and student 
teams of varying sizes at IU. Virtual Bethel’s success has inspired the 
integration of MAS students and faculty in the virtual re-creation of 
several environments and time periods in and around Indianapolis. 
The collaboration of community members, historians, preservation- 
ists, librarians, student game developers, and 3D artists realizes a 
new opportunity to develop exciting experiences that are authentic, 
accurate, and informative, both inside and outside of academia. 

We did not just scan the Bethel AME church, we did not just doc- 
ument the space, we re-created it. Furthermore, we are preserving it 
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in a medium that will far outlive the physical church or anyone on 
the team. We embedded within the VR environment access to more 
history than the real space could ever provide, separate from any 
single historian, member, or moment. We made it possible to add 
and amend new content to the Virtual Bethel database at any time. 
We believe projects like Virtual Bethel are redefining what the pres- 
ervation of an endangered cultural heritage site means. The scanning 
of 2D and 3D objects is the first step in a much larger preservation 
pipeline, one in which an audience readily accesses a space that no 
longer exists, listens to voices that can no longer be heard, and holds 
artifacts that no longer can be held. Future audiences will demand 
to interact with and understand history on their own terms, while a 
new niche of VR curators will initiate preservation projects, provide 
access, and steward the experience. 
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Chapter 4 


Using 3D Photogrammetry to Create 
Open-Access Models of Live Animals: 
2D and 3D Software Solutions 


Jeremy A. Bot and Duncan J. Irschick 


Abstract 


Novel technological solutions emerging over the past five years have 
made it possible to consider previously unheard of re-creations of 
the world as it exists in its stunning, full-color, 3D topography. The 
potential value of the rendering of real animals for science, conser- 
vation, education, and story-telling is substantial. For scientists, 3D 
models of live animals provide valuable data for testing theories on 
body shape and movement, and they represent “avatars” of actual 
specimens for further analysis. For conservationists, the ability to use 
novel technological solutions such as virtual reality (VR), augmented 
reality (AR), or gaming applications to present real-life animals 
opens new doors for reaching the public. For educators, the ability 

to tell stories around a specific animal, instead of a generic animal, 

is significant. Drawing on our work with the Digital Life Project, we 
describe our process for using open-source software to recreate liv- 
ing animals in 3D, from photocapture to animation. 


Introduction 


he past decade has witnessed an upsurge in efforts to digitally 
preserve the world, such as 3D scanning of buildings (cyark. 


org) and corals (thehydro.us). Some methods for 3D scanning 
include magnetic resonance imaging (MRI), computed tomography 
(CT), laser or white light scanning, and photogrammetry (Huising 
and Gomes Pereira 1998; Bythell, Pan, and Lee 2001; Gignac and Kley 
2014). Of these methods, photogrammetry has emerged as a flex- 
ible tool that can be used for various research applications (Debevec 
et al. 1998; Bythell, Pan, and Lee 2001; Dai and Lu 2010). Unlike the 
other methods listed, photogrammetry does not require expensive 
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hardware or software (Weinberg et al. 2004; Linder 2009; Falkingham 
2012). 

This chapter examines the practice and art of generating 3D 
models of live animals using photogrammetry—the science of deriv- 
ing measurements in 3D space using photographs (figure 4-1). In 
particular, we focus on the technical workflow using open-source 
software (i.e., computer programs that are distributed with the origi- 
nal source codes, which allow them to be read and edited in their 
uncompiled state). 


Fig. 4-1. A green sea turtle is scanned 
using photogrammetry software, Re-creating live animals as full 3D models by means of photo- 


romogeied, Teauled, anid annals: grammettry is not simple. For entertainment purposes, a 3D artist 
creates a generic animal model using multiple reference photos from 
various animals, modeling in as much detail as necessary. For pur- 
poses of conservation, science, and education, the 3D artist is under 
more constraints as they must be faithful to the original details of a 
single animal. It is also important to realize that most 3D scanned 
models represent a single snapshot, with the subject frozen in time. 
This is an issue as animals are rarely static. The ability to record and 
demonstrate the movement of a living animal enhances storytelling 
elements and can provide new tools for scientists and educators. For 
example, a recording of a dinosaur in motion, if sucha thing were 
possible, would expand our understanding of how these creatures 
moved. As behaviors affect structures, this helps us better under- 
stand the animal’s underlying anatomy, and even gives hints to the 
environments they traversed. Thus, in the review of our technical 
workflow, we also discuss the methods and mindset that we used for 
re-creating movement. 

In this paper we first focus on the process of using 3D photo- 
grammetry for the photocapture of objects. Second, we discuss the 
use of this method and other aspects of photography to photocap- 
ture live animals in different settings (e.g., field, laboratory). Third, 
we show how open-source software (e.g., Blender) can be used to 
reconstruct the resulting 3D meshes and then render a model in its 
final forms to create a VR- and AR-ready “asset” (Raitt and Minter 
2000; Yirci 2008). 


Uses for 3D Models of Living Animals 


Lifelike 3D models of animals (i.e., those that appear in death much 
the same as they do in life) are valuable for several reasons. In many 
cases, such as with mammals and birds, scanning a dead specimen 
is very different from scanning a live specimen, both in terms of 
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the process and the result, whereas in other cases, the distinction 
will not be obvious. First, these 3D models can represent 3D digital 
voucher specimens. Traditionally, scientists collect live specimens 

in the field, euthanize them, preserve them with fixatives, and then 
deposit them in museums. Using photogrammetry, researchers can 
scan live specimens in the field or the lab and then create 3D models 
that are connected with metadata (e.g., museum accession numbers) 
for scientific analyses. Second, the 3D models can be used for com- 
putational modeling analyses, such as testing theories of how sea 
turtles or sharks swim. Third, the models have various educational 
uses, ranging from virtual and augmented reality (VR and AR) to 
3D printing to demonstrate animal form and function (Blagoderov 
et al. 2012). Fourth, the models serve as a powerful tool for animal 
conservation programs. In the same ways that photography pio- 
neered new ways of understanding animals, the use of VR and AR 
has similar potential to facilitate the visualization of live animals 
and demonstrate their uniqueness. Finally, the 3D models have po- 
tential commercial value, as unique 3D models from animals allow 
more specific content for various kinds of work, such as commercial 
movies or video games. 


Capturing and Processing 3D Data 
Using Photogrammetry 


Combined with computer software, digital photos can be used to 
generate a 3D polygon mesh and surface color texture files. Many 
popular photogrammetry software solutions can process all steps in 
the model-making process, from the capture of initial photos to the 


Software for Photogrammetry and 
3D Mesh Manipulation 


Photogrammetry software: 


RealityCapture by CapturingReality, 
https://www.capturingreality.com/ 
PhotoScan by Agisoft, http://www.agisoft.com/ 


Open-source software tools that also handle various portions of 
the photogrammetry processing 


<= 


isualSFM, http://ccwu.me/vsfm/index.html 
MeshLab, http://www.meshlab.net 

Bundler, http://www.cs.cornell.edu/~snavely/bundler 
CMVS, http://www.di.ens.fr/cmvs/ 

COLMAP, https://colmap.github.io/ 

VE, https://www.gcc.tu-darmstadt.de/home/proj/mve/ 
VS-Texturing, 
https://www.gcc.tu-darmstadt.de/home/proj/texrecon/ 
OpenMVG, https://github.com/openMVG/openMVG 
OpenMVS, http://cdcseacave.github.io/openMVS 
Theia, http://www.theia-sfm.org 

Meshroom, https://github.com/alicevision/meshroom 


M 
M 


production of final textured 3D mesh (sidebar at left). 


Technology Overview 

The first step in scanning an object with photogram- 
metry is to take multiple photos of the subject at vary- 
ing angles. Next, the computer software generates a 
3D mesh by finding similarities between sets of pho- 
tos. The matched patterns in each photo are triangu- 
lated, resulting in the conversion of pixels into points 
in 3D space. The process enables us to use these point 
cloud data to construct a polygon mesh. Having cal- 
culated the coordinates of the points and the positions 
of the camera, we can project the original photos onto 
the surface of the mesh to create a color texture map. 
Because photogrammetry relies on the ability to iden- 
tify similar patterns between each photo, the process 
works best with detailed, nonreflective surfaces that 
are viewed under consistent lighting. For example, it 
is problematic to reconstruct shiny objects or objects 
with no obvious landmarks (e.g., a white sheet) with 
3D software.’ 


! The process of using photographs and videos to create 3D models via 
photogrammetry has been covered in other publications (e.g., Falkingham, 2012; 
Bagersad et al., 2017) 
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Fig. 4-2. Frog surrounded by a 
multicamera rig built by the Digital 
Life Project. Photograph by Christine 
Shepard. 


Single Camera 

Because it is possible to reconstruct only what the lens can see within 
its viewing angle, the use of a single camera is best suited for situa- 
tions in which the subjects being sampled are static, such as architec- 
ture, sculptures, or other inanimate objects. A static object allows the 
photographer to move around the subject, taking multiple pictures 
from various viewing angles, without having to worry that the sub- 
ject will move between photos. For most living animals, therefore, 
the use of a single camera is unsuitable. When a subject moves with 
respect to the background while photos are being captured in se- 
quence moves, the patterns in the photos will not align properly. If 
there are misalignments, the resulting 3D reconstruction will have 
inaccuracies, resulting in a lack of resolution or the presence of visual 
artifacts or noise. Further, lighting quality around a subject, both in 
the field and the laboratory, can vary considerably. Thus, the effec- 
tive use of photogrammetry as a tool requires some skill in lighting 
and basic photography. 


Multiple Cameras 

To capture moving objects (e.g., animals), one method that the Digi- 
tal Life* team has used is a multicamera rig (figure 4-2). Multicam- 
era rigs can be configured in many sizes and shapes, and they have 
been most widely used for human 3D imaging, such as the xxArray 
system by artist Alexx Henry in Los Angeles. A typical multicamera 
rig consists of 4 to 30 cameras on some form of fixed system, such 
as tripods or sets of 80/20 metal rods, with all of them pointing 


* http: / /digitallife3d.org 
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toward a central area. The cameras are then synchronized to wireless 
triggers or wires, which ensures that all of them take a photo at the 
same time. An alternative method is the use of synchronized video 
with a common motion (e.g., a clapper or a ball drop in view of all 


cameras). 


Our focus has been on creating multicamera rigs for work with 
live animals that vary in shape, size, and behavior. The Digital Life 
team focused on creating four different kinds of multicamera rigs, 
which together represent the Beastcam technology platform. These 
rigs are the handheld Beastcam, the Beastcam MACRO, the Beastcam 
ARRAY, and the Beastcam STAND. The handheld Beastcam is a four- 
camera system designed in part by Kasey Smart, Dylan Briggs, and 
Duncan J. Irschick. The system works through the synchronization of 
four small cameras on flexible arms that are triggered through wires. 
The Beastcam MACRO was designed by Trevor Mayhan for creating 
3D models of small animals ranging in size from about 1 to 5 inches 


Multimedia Data File Format Definitions 


DNG: Photos taken with our cameras use this raw Adobe 
digital negative format. https://helpx.adobe.com/photo- 
shop/digital-negative.html 


OBJ: We use this open-source file format developed by 
Wavefront Technologies, a popularly used standard for 
storing 3D mesh data. It includes texture coordinate 
information (also known as UVs) that maps a 2D texture 
to the 3D model. When combined with MTL files, the 
OBJ file can manage material properties and link to the 
texture image files that have been applied to the mesh. 
In our process, we export our scans in this format as it is 
compatible with nearly all 3D animation software pack- 
ages. However, it does not support skeletons, skinning, or 
animation data. http://www.cs.utah.edu/~boulos/cs3505/ 
obj_spec.pdf 


PNG: For our textures, we use this open standard lossless 
image format that supports 24 bit color depth with RGB 
or 32 bit RGB + Alpha. Commonly used for textures in 
3D animation because of its color depth and lossless data 
compression and on the web as recommended by the 
World Wide Web Consortium (W3C). http://www. libpng. 
org/pub/png/ 


BLEND: This open-source file format is the native file type 
for Blender, our core 3D open-source animation tool set 
protected under General Public License v2. The format 
supports mesh geometry, UVs, materials, skeletons, skin- 
ning, and animation. Blender has been slowly growing in 
popularity since the free version was released in 2003. 
Currently, many game engines and 3D websites are di- 
rectly supporting Blender’s file format. We use .blend 
files throughout our 3D modeling and animation process. 
https://docs.blender.org/manual/en/dev/data_system/ 


in body length, such as small frogs or reptiles. The 
Beastcam ARRAY, developed by Zachary Corriveau, 
can hold more than 40 cameras and was funded in 
part to create 3D models of small reptiles. This system 
is best suited for small to mid-sized animals ranging 
from about 3 to 10 inches in length. Finally, the Beast- 
camSTAND system was created by Michael Perriera 
originally for 3D models of houses, but the utility of 
this system for large animals soon became apparent. 
Each of these systems is field-portable. The inclusion 
of scale bars next to the model allows the user to re- 
construct the scale of the specimen. 


Aligning Images to Create an 
Accurate 3D Model 
Especially with live animals, coverage of photos is 
typically insufficient to capture in detail every portion 
of an animal. The capture of additional images over 
occluded areas with a handheld camera can help to 
remedy this problem, even when the original animal 
has moved from the position in which all the cameras 
were synchronized. In the case of sea turtles, we used 
two scans (dorsal and ventral sides) along with indi- 
vidual images to create the full 3D model (figure 4-3). 
However, because the animal was often in a different 
pose when the dorsal and ventral sides were scanned, 
additional work was necessary to integrate the pho- 
tos/scans into a single model. 

The animal poses within the individual scans 
are adjusted so that the meshes align. This is accom- 
plished by finding corresponding landmarks between 
the portions. These landmarks serve as reference 
points, allowing the photographer to scale the mesh 
portions so that they match in size. Once the rigid 
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elements of the animal have been used to bring the portions to a 
consistent scale, the poses of the extremities are aligned with one an- 
other. Rather than arbitrarily adjusting the points of the mesh, a tem- 
porary motion rig (used to allow the model to move) is placed within 
each portion so that limbs can be rotated into place. This procedure 
helps reduce the risk of distortions, thus preserving the proportions 
and anatomical structure of the animal modeled. Finally, the aligned 
mesh portions can be manually stitched together or used as a tem- 
plate onto which a mesh can be drawn. 


Fig. 4-3. The dorsal (left) and ventral (right) sides of a green sea turtle are captured in two separate scans using our multi- 
camera setup. The scans are processed separately within the Reality Capture (photogrammetry software), then brought into 
the Blender animation software for merging and cleanup. 


Reducing the Polygon Count of 3D Mesh 


The 3D mesh reconstructions from our photogrammetry processing 
have a high number of polygons, which should ideally be reduced 
to reduce processing time. The models exported from our photo- 
grammetry software are in the OBJ file format, contain 3 to 6 million 
triangle polygons, and include fine surface details mixed with mesh 
artifacts typically caused by surface reflections in the source photo- 
graphs (figure 44). Using our current camera type and configura- 
tion, we have found that meshes exported in excess of 6 million poly- 
gons add no useful details, but rather result in more unusable scan 
noise. Though many of today’s computers can display these high- 
density 3D models, the processing performance of different systems 
varies widely. 


Traditional Method: Simplifying Mesh Using Edge Collapsing 
Processes traditionally used for reducing the number of polygons 

in a reconstructed mesh involve combining / collapsing the existing 
polygon edges (Yirci 2008). Many of these simplification tools have 
automated processes that will decimate the mesh to a specified re- 
duction percentage. 
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Fig. 4-4. This 3.2 million polygon mesh reconstruction of a pixie frog’s face shows the eyes 
and a nostril. Surface reflections on the skin and eyeball have caused some noise artifacts that 
appear as sharp bumps, cavities, or wrinkles on the surface. 


The decimated mesh is less dense than the original photogram- 
metry reconstruction (figure 4-5). Though the number of polygons in 
the decimated mesh is more manageable, the irregular, triangulated 
structure of the mesh does not produce the best results when used 
with mesh deformers common in 3D animation (e.g., skinning, lat- 
tices). Instead of using this approach, we have adopted an alternative 
method. 


Fig. 4-5. The 3.2 million polygon mesh of the frog shown in figure 4-4 would be a challenge to work with on most computer 
systems (left). Using Blender’s Decimate modifier, we reduced the mesh to 1/50th of its original polygon count, resulting in a 
workable number of 64,000 polygon triangles, which can be viewed or edited on even the most modest of computer setups. 
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Our Preferred Method: Simplifying Mesh While Optimizing 

the Mesh for Deformation 

Converting the geometry and textures to an animation-friendly 

format is a task often left to “3D artists,” as it typically involves a 

manual process of 3D mesh reconstruction commonly referred to 

as retopologizing. We have two primary goals when simplifying the 

dense polygon mesh in preparation for animation: 

1. Reduce the polygon count, enabling better 3D performance of 
the model in real time while maintaining the accuracy and high- 
frequency detail of the original scan (via “normal map” texture 
files). 

2. Structure the polygon edges in a way that allows for better defor- 
mation when the mesh is bound to a skeleton for animation. For 
instance, in figure 4—6, the benefits of using quad polygons (right) 
over triangular polygons (left) are clear. 


Fig. 4-6. A randomly triangulated mesh (left) is not based on the function of the underlying anatomy and therefore produces 
undesirable wrinkles and bumps in the mesh. A retopologized quad mesh (right) follows the flow of the limbs, with edge loops 
lining up with the creases in deformation, thus producing fewer undesirable distortions. 


A reduced mesh can contain non-manifold geometry or 
intersecting polygons that occur when the decimate processes 
attempt to reduce natural high-frequency detail or surface noise 
generated in the photogrammetry processing. 

A mesh that has been retopologized to include primarily quad 
polygons can be structured so that the polygon “edge flow” aligns 
with the direction of deformation (figure 4-7). These quad meshes 
are common in animated 3D models used in film, television, and 
video games (Raitt and Minter 2000). 
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Fig. 4-7. This quad mesh of 10,000 polygons shows an edge flow that resembles the underlying muscle structure of a human. 
The arm (left) shows a slight twisting of the polygons as they move down over the forearm. The blue strip on the forearm 
resembles the flow of the brachioradialis muscle, which allows the forearm to twist inward and outward. The face topology 
(right) resembles the muscles that raise the eyebrows, surround the eyes for squinting, and circle the mouth for retracting for 
a snarl or pulling at the corners of the mouth for a smile. Model by Angela Guenette for the Blenderella modeling class,3 used 
with permission and licensed via CC BY 3.0.45 


Retopologizing is traditionally a manual process, though soft- 
ware tools are now available to assist this process by allowing a 
user to change the mesh structure by drawing curves on the surface. 
The algorithm guides the reconstruction of the mesh edges into a 
new quad mesh topology that follows these curves. Commercial 3D 
sculpting software packages that include automatic retopologizing 
include ZBrush by Pixologic, 3D Coat by Pilgway, Autodesk’s Mud- 
box, and The Foundary’s Modo. 

We use Instant Meshes,’ an open-source software solution, to 
begin the process of retopologizing our high-density meshes. With 
this software, we can create guide curves and export a reduced mesh 
consisting of mostly quad polygons with an edge flow that matches 
our guides. These guides not only simplify some of the cleanup 
process, but also give us quick edge flow results that align with the 
shape of the scanned animal (figure 4-8). 


3 https: / / ponderstudios.com/2017 /09/07/advanced-character-modeling-workflow / 
* https: / / creativecommons.org /licenses/by /3.0/ 

5 https: / / creativecommons.org /licenses/by /3.0/ 

6 https: / / github.com/ wjakob/instant-meshes 
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Fig. 4-8. In this comparison of the topology of the top of the frog’s head, we can see that the original dense mesh of 3.2 
million polygons (left) has been retopologized using Instant Meshes into an isotropic quad mesh of 60,000 polygons with an 
edge flow that preserves the silhouette (right). 


In addition to the reduction of polygons and better edge layout 
for skeletal animation,’ the resulting edge flow simplifies the process 
for re-creating the UV maps for the textures. 


Reconstructing Mesh Normals and UV 
Coordinates 


It is important to accurately reconstruct mesh normal and UV coordi- 
nates as these elements dictate the overall color map of a 3D model, 
as well as ensure that the model can be effectively used in a wide 
variety of platforms. 


UVs 

The term UVs refers to UV maps or texture coordinates. The letters 
U and V denote the axes of a plane, since X, Y, and Z are used for the 
coordinates in the 3D space. For a 2D image texture to be applied to 
a 3D object, each vertice of a3D mesh contains a pair of numbers (Us 
and Vs) that represent the location at which that particular vertice 
would plot ona scale of 0 to 1 on a square plane. The Us and Vs 
refer to the height and width coordinates. A UV map is visualized 

as a wireframe cage flattened into something that resembles a 2D 
clothing pattern or animal pelt. Marked “seam” edges enclose these 
pieces (UV “islands,” as Blender calls them). Many 3D software 
packages allow the user to create seams on the mesh (the shores 

to the metaphorical islands) by selecting edges in a 3D viewport 

and then marking the edges as seams (figure 4-9). Next, running 

an unwrap function cuts the mesh along its marked seam lines to 
flatten the mesh into a 2D plane. This function attempts to limit the 
stretching of the mapping by ensuring that the relative area and 
general shape of each UV polygon is similar to that of the source 
polygon on the initial 3D model. 


’ Wikipedia defines skeletal animation as “a technique in computer animation in 
which a character (or other articulated object) is represented in two parts: a surface 
representation used to draw the character (called skin or mesh) and a hierarchical 
set of interconnected bones (called the skeleton or rig) used to animate (pose and 
keyframe) the mesh.” 
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Fig. 4-9. UV texture coordinates on top of the color texture image with seams outlined in red (left) and 3D turtle model with 
edges marked as seams for automatic layout of UVs using Blender’s UV unwrap function (right). 


Most photogrammetry software has the capability to automati- 
cally generate the UVs for the resulting 3D mesh scan. However, 
these auto-generated seams can result in a map with fragmented tex- 
ture files (see figure 4-10, left image). 


Fig. 4-10. The auto-generated UV maps on the scanned mesh are often quite fragmented, and while useful initially, they can 
be very difficult to work with when texture painting or manipulating the texture file using an external image-editing software 
(left). Seams are created on the retopologized frog model by selecting edges and marking them (middle). The color texture 
information can then be transferred from the original mesh scan to the new retopologized mesh via a baking process, which 
results in a texture file that is much easier to work with in the texture-editing process, though it may result in some unused 
texture space (right). 


As the retopologization creates a new mesh, it is necessary to 
create a new UV map. A newer, cleaner edge flow will allow for 
seams to be easily marked using edge loop selection shortcuts in 
the animation software (figure 4—11). The resulting UV will usually 
contain dozens of UV islands, instead of the hundreds or thousands 
of islands that result from the original photogrammetry scan. 
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Fig. 4-11. The cleaner edge flow that can result from retopologization can also be used to cleanly trim unnecessary geometry 
from the scan. These split edges now create a boundary that makes it possible to select points on the mesh section to 

be trimmed, and then the “Select Linked” function will grow the selection until it hits the split edge boundary (middle). The 
selected geometry can now be deleted, resulting in a trimmed mesh with a less jagged silhouette (right). 


Sometimes the new UV islands, if not organized efficiently, can 
result in some large gaps between the islands; these gaps represent 
unused texture space. Therefore, it is important to compare the origi- 
nal textured photogrammetry scan with the newly textured retopolo- 
gized mesh to see if there is any loss in resolution or texel density® 
(i.e, number of pixels per square meter in real life). 


Normals 

When describing a 3D mesh, normals are often used to describe 

the facing direction of the polygon vertice, edge, or face. A normal 

is a vector that is typically 
perpendicular to each polygon 

face (figure 4-12). It is important to 
have normals facing in a consistent 
direction (preferably pointing 
outward), and they can be adjusted 
manually or automatically. Blender 
has some features that allow 

the normals to be automatically 
recalculated, smoothed, or flipped. 
Normals should be adjusted before 
the baking of the “normal map” in 
the next section (conversion of high- 
frequency mesh detail into a 2D 
image texture). 


Fig. 4-12. The face normals (red) show a vector that is perpendicular to each 


face of the retopologized mesh. 


§ An infographic on texel density is available at https: / / www.artstation.com/ 
artwork/qbOqP. 
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Recovering High-Frequency 
Geometry Details 


The mesh retopologization reduces the polygon count and improves 
the edge flow, but the reduction removes the high-frequency details 
of the mesh. There are a few different ways to recover this detail. The 
following method brings back the fine details, but applies them to 
our more deformation-friendly retopologized mesh. 


Multiresolution and Shrinkwrap Modifiers to 

Recapture High-Frequency Details 

Like other 3D animation software packages (e.g., 3D Studio Max), 
the Blender 3D animation software package contains modifiers that 
allow us to alter our geometry without losing the original input 
mesh data. In most cases, these modifiers can be permanently ap- 
plied to the mesh or cumulatively stacked, maintaining the original 
base mesh information and modification history so that we can track 
our progress. We use two modifiers for transferring the high-fre- 
quency scan data into our retopologized mesh. The Multiresolution’ 
modifier subdivides our mesh into levels, increasing the polygon 
count exponentially with each subdivision. This modifier can also 
store mesh edits that have been applied at the various subdivision 
levels. The Shrinkwrap” modifier adjusts the affected mesh by mor- 
phing the surface to match that of another target mesh. 

To retrieve the high-frequency geometry from our original scan, 
we use the Multiresolution modifier to first increase the polygon 
count of our retopologized mesh. The Multiresolution modifier adds 
subdivision levels where we can store the high-frequency details 
from our original scan, and it also gives us the ability to use Blend- 
er’s geometry sculpting tools to smooth out some of the high-fre- 
quency surface noise or to add details back into the mesh that were 
lost during the photogrammetry processing. Pressing the Subdivide 
button within this modifier adds new subdivision levels; we can then 
use the Preview or Sculpt slider values to set the number of subdivi- 
sion levels that we need (figure 4-13). 


Fig. 4-13. The Multiresolution modifier in Blender receives an input mesh (far left) and is able to add subdivision levels, 
increasing the polygon count in steps to match or exceed that of our original 3.2 million polygon scan, eventually providing 
enough point data to capture the high resolution data from the original photogrammetry scan. 


° https: / / docs.blender.org / manual /en/ dev / modeling / modifiers / generate / 
multiresolution.html 

https: / / docs.blender.org / manual /en/dev/ modeling / modifiers / deform / 
shrinkwrap.html 
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With the polygon count of our retopologized mesh increased, we 
can now use Blender’s Shrinkwrap modifier to match the original 
photogrammetry mesh. As the name implies, it shrink-wraps the 
modified mesh by pushing out or pulling in the points of our now 
subdivided retopologized mesh to match that of the original scanned 
mesh (figure 4-14). The Shrinkwrap modifier has an option to auto- 
matically perform this point-to-surface matching, using projections 
of the surface normals in the positive or negative directions in the 
Project mode. The new surface position information from the Shrink- 
wrap modifier can then be permanently applied to, and stored in, the 
Multiresolution modifier at a chosen subdivision level by clicking the 
Apply button for the Shrinkwrap modifier in the stack. The Apply 
button will make the Shrinkwrap modifier’s effects permanent by 
storing them within the Multiresolution modifier (the next modifier 
in the stack). Because this change is made permanent, the Shrink- 
wrap modifier is no longer needed and is automatically removed. 
The Multiresolution modifier stores these modifications within the 
subdivision level specified in the Preview numerical slider. 


Fig. 4-14. The retopologized mesh with a Multiresolution modifier added at only four subdivision levels (left). While previewing 
four subdivision levels, a Shrinkwrap modifier is also added to the mesh, using the original 3.2 million polygon photogrammetry 
mesh as a target (right). 


With the high-frequency details applied to the retopologized 
mesh via the Multiresolution modifier, Blender’s sculpting tools can 
now be used. Unless the Multiresolution modifier’s Apply Base but- 
ton is pressed, the modifier will store the sculpting information and 
keep the base retopologized mesh unaltered. Though the informa- 
tion in this modifier can be accessed only from the Blender animation 
software, we can export the retopologized mesh with all the captured 
high-frequency details and sculpted updates at whatever subdivision 
level we choose, depending on the level of detail required for our 
specific needs. 
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Normal Maps: Storing High-Frequency Surface 
Detail Via Image Texture File(s) 
If a lower polygon count is required, a normal map is commonly 
used to “fake” high-resolution surface detail. The normal map is 
a texture applied to the surface of the mesh that interacts with the 
lighting to create the appearance of high-resolution detail without 
the need for a high polygon count. This is significant for reducing the 
file size of the 3D asset and improves performance when the model 
is animated. In most cases, the differences between actual geometry 
and normal-mapped details are imperceptible to the average user. 
The normal map works in combination with the mesh normals 
(direction in which each polygon face faces) in order to accurately 
calculate the way light interacts with the surface (figure 4-15). There- 
fore, if the mesh changes, the normal directions may change, and 
noticeable problems develop in the way in which light interacts with 
the normal-mapped surface. When dealing with organic meshes, the 
development of visible (hard) polygon edges is the most common 
problem. 


Fig. 4-15. The retopologized mesh (left) has a normal map image file (center) applied in order to change the way that light 
affects the surface, which creates the perception of high frequency polygon detail for users without increasing the total 
number of polygons (right). 


The creation of a normal map involves two mesh objects: a 
source mesh that contains the high-frequency mesh details, and a 
lower polygon target mesh that will receive the normal map with the 
high-frequency details (figure 4-16). For this task, we work within a 
Blender file that contains a single mesh object that has the high- and 
low-resolution details required to make a normal map stored in its 
Multiresolution modifier. To create source and target objects, we du- 
plicate the mesh object. We now remove the Multiresolution modifier 
from one of these objects, but not before determining the polygon 
density that best fits our destination platform (e.g., games, film, 
web). Testing will be necessary to determine the best performance 
for a particular platform, but generally a real-time graphics engine 
requires a lower number of polygons, while video content functions 
can handle a higher number of polygons. 

On our lower resolution target mesh object, we use the Preview 
slider on the Multiresolution modifier to choose a subdivision level 
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that works for our needs and make this permanent by clicking the 
Apply button. To receive the normal map information, we create a 
new texture for the target mesh object. The Bake" options are found 
in the Render Properties panel. A normal map is baked by selecting 
both source and target meshes, setting the Bake Mode to Normals, 
and clicking the Bake button. Since Blender currently has more than 
one render engine, there are two methods to bake these textures. In 
Blender’s current state, we have found that the Blender Internal ren- 
der engine is faster than Blender’s Cycles render engine. These nor- 
mal maps are saved into a PNG image file format and applied to the 
low-resolution target mesh in the object’s materials inside of Blender, 
or exported for use in other 3D applications. 


Source Mesh 
high frequency 
w/ Multiresolution 


Normal Map 
image texture 


BAKE 
a NORMALS a 


Target Mesh 
lower resolution 
w/o modifiers 


Target Mesh 
lower resolution 
w/o modifiers 


Fig. 4-16. The high-frequency source mesh and lower resolution target mesh are baked to 
create a normal map texture that is applied to the lower resolution mesh to capture the high- 
frequency details without the overhead typically needed for meshes with high polygon counts. 


Rigging and Animating the 
Retopologized Mesh 


The final steps for bringing our model to life involve the applica- 
tion of movement to a 3D rig. Because the details of creature rigging 
and animation are software specific, we will cover only basic theory, 
while pointing out considerations that apply to animal scans, par- 
ticularly those that require the merger of multiple meshes. 


Creation of the Skeletal Rig 

Animation of a polygon mesh requires the creation of a skeletal rig. 
These rigs are composed of a hierarchy of bones (also known as joints 
in Autodesk’s Maya) that give the polygon mesh a site to attach it- 
self. When the joints are animated, each vertice of the mesh follows 
the influencing bone(s) based on a weight value, causing the mesh to 
deform. For example, a point that is weighted 100 percent to a single 
bone will follow the influencing bone by maintaining its relative 


1 Texture baking in Blender: https: / /docs.blender.org / manual/en/dev/ render / 
blender_render / bake. html 
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positioning to the influence (i.e., the force applied to that part of the 
skeleton), while a vertice that has its weight split between two bones 
(50 percent to each) will maintain a position halfway between its 
relative positioning to each of the two influencing bones. 

The process of attaching the mesh to the skeleton is commonly 
referred to as “skinning.” Most 3D animation software has a default 
or automatic bind process that can be useful for quickly binding the 
mesh to a skeleton with minimal manual weight “painting” work re- 
quired. The automatic process looks for bones that are closest to each 
vertice and automatically assigns influencing weight values to each 
vertice. To customize the results of the automatic binding algorithms, 
extra bones can be added to the hierarchy (figure 4-17). Though a 
skeletal rig can appear similar to a biological skeleton, it is typically 
different; the rig’s primary goal is to produce the most accurate de- 
formation results on the mesh geometry during animated motion. 


Fig. 4-17. Based on anatomical study and an understanding of mesh deformation via a skeleton (“Armature” as Blender calls 
it), bones are placed inside of a turtle mesh. The vertices of the mesh are bound to the skeleton, with each bone’s influence 
over the mesh displayed as a heat map. Additional bones are added inside the shell in order to prevent the turtle’s flippers 
from affecting the shell, keeping the shell rigid (left). The retopologized edge flow greatly simplifies the adjustment of these 
influence “weights,” as demonstrated by the edge rings influenced by the turtle’s neck bone (right). 


Fig. 4-18. Two scans capture the turtle in two halves, each in a unique pose (left). The purple mesh is displayed bound 
to a purple skeleton, while the gray mesh is bound to the gray skeleton via an automatic skinning process and aligned by 
manipulating the skeletons, which deform the two meshes (right). 
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In cases where multiple scans must be merged into a single 
model, a skeleton can be used to pose both halves of the specimen 
while preserving the proportions of the animal (figure 4-18). Instead 
of arbitrarily adjusting the points of the mesh manually, the halves of 
the mesh are posed using the skeleton to meet at an in-between pose. 
The alignment of the skeleton and mesh surfaces assists in the vali- 
dation of the skeleton placement and mesh skinning. 


Animation 

Working with video of animal movements, we typically use the 
rigged /animatable model to create a repeating / cycling locomotion 
(e.g., Lasseter 1987; Kerlow, 2009). For a land creature, this involves 
a walk cycle. For most of our marine animals, we create swim cycle 
animations. Though our goal is to create accurate motions, animal 
movement is dynamic and not easily captured in a single animation. 
We are currently working on processes to more accurately apply 
animations to our models, increasing the level of detail and decreas- 
ing the time needed to create these animations. When working with 
animals of a similar structure, we aim to repurpose and refine previ- 
ously created animations and workflows. 


Conclusion 


In this paper, we have provided an overview of the technical work- 
flow of rendering live 3D animals using photogrammetry. Our 
workflow has value as it enables 3D artists, scientists, and educators 
to recreate complex 3D geometries in a scientifically accurate man- 
ner. The value of the full-color, accurate, and fully rigged 3D animal 
models is that that they represent valuable starting points for many 
scientific investigations, as well as educational use for demonstrating 
biodiversity and animal form and function. 
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Chapter 5 


What Happens When You Share 3D 
Models Online (In 3D)? 


Thomas Flynn 


Abstract 


As 3D content capture and production become increasingly acces- 
sible and affordable, more cultural institutions are contemplating the 
use of this format alongside established media such as 2D photogra- 
phy. This chapter addresses the possibilities and value of publishing 
cultural heritage-related 3D data online, in a real-time interactive for- 
mat. Using existing projects and initiatives published on the Sketch- 
fab platform by way of example, the chapter provides an overview 
of 3D within the cultural heritage sector. 


Introduction 


s the cultural heritage lead at Sketchfab, much of my work 
involves promoting the use of 3D within the cultural sector, 


including libraries, archives, and museums. This includes 

introducing organizations to the concept of 3D digitization if they do 
not already have a program in place and extolling the benefits of put- 
ting 3D models online if they are already creating digital 3D files. 

This chapter will draw heavily on my personal experience advis- 
ing cultural heritage professionals with regard to 3D and producing 
3D content for national institutions; I will offer examples of how 
individuals and institutions within the Sketchfab community are 
using 3D to achieve their goals. While I heartily recommend a trial 
of Sketchfab to display 3D, regardless of which browser-based 3D 
viewer is used,' these examples highlight the value of dynamic, on- 
line interaction with 3D content. 

A broad theme to keep in mind in discussing the creation and 


1 Other open-source and commercial viewers are mentioned in the following pages. 
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use of digital 3D models of cultural objects and spaces—especially 
online—is one of experimentation and, to a certain degree, skepti- 
cism. Since I began working in the realm of 3D for cultural heritage 
in 2014, I have seen the boom in 3D production matched with a 
boom in conferences, think tank activities, and online discussions 
about various aspects of the technology. At the same time, there is 
still some skepticism in a variety of fields about the value of 3D as 
more than just a fad or experimental technique. 


Defining 3D 


The term 3D is often understood to describe various non-interactive 
and interactive experiences, including 2D renders (i.e., images, 
video) of 3D data, animated turntable presentations, 360-degree 
panoramas, all the way to complete virtual or augmented reality 
experiences. 

Even when we say something like 3D data, there are questions. 
Are we talking about vector or raster data? Point clouds or meshes? 
Textured or untextured? Animated or static? Raw or edited data? 3D 
scanned or computer generated? 

With regard to the cultural and historical 3D content on Sketch- 
fab, the vast majority of 3D models are surface 3D models produced 
via photogrammetry and, to a lesser degree, structured light, laser 
scanning, and computed tomography. A relatively small portion of 
the 3D models are born digital (i.e., a “3D representation of an item 
that may not have a specific real-world counterpart ... created using 
digital imaging or drafting software rather than scanning” [DeVet et 
al. 2018)). 

The variation in understanding may result in part from personal 
experience. Somebody who has grown up with interactive games 
and computer-generated movies may expect the term 3D to apply 
to the experience of 3D, whereas somebody who has been trained to 
use 3D imaging workflows in professional work may generally con- 
sider 3D to refer to quantifiable data. Understanding the variations 
in perceived meanings may help better define the best presentation 
method for 3D models for a given audience. 

Based on my own experience with 3D, I would go so far as to 
suggest that the term 3D means real-time interaction, that it indicates 
we are in control of our experience of the model or data and able 
to move to a different perspective at will. The ability to manipulate 
and not just spectate is a simple yet key reflection of how we explore 
physical objects in the real world, and digital 3D allows this to hap- 
pen virtually. To experience the full value of true 3D is through dy- 
namic, personal interaction. 

Being explicit in our definition can help us avoid confusion as 
the cultural heritage community establishes metadata standards 
(e.g., the Community Standards for 3D Data Preservation [CS3DP] 
Project,’ the International Image Interoperability [IIIF] Community 


2 http:/ /cs3dp.org 
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Group on 3D®*) and pursues ongoing digitization projects and col- 
laborations. By considering 3D data separately from 3D experiences 
and services, we can perhaps build stronger and longer-lasting use 
cases for 3D in the cultural heritage sector. 


Growing Popularity of 3D 
in the Cultural Heritage Community 


With more than 650 museums using Sketchfab to share some of their 

collections in digital 3D, plus thousands more libraries, scientific 

organizations, individual researchers, imaging experts, and hobbyist 

3D scanners using it, Sketchfab has become a go-to tool for the cul- 

tural heritage sector to display and disseminate 3D data online. As of 

October 23, 2018, there were 74,262 3D models in Sketchfab’s Cultur- 

al Heritage & History category,’ 12,248 of which are downloadable 

under one of the following Creative Commons licenses: 

¢ CC BY—Attribution: Depending on the non-commercial (NC), no 
derivatives (ND), and share alike (SA) choices, others may share, 
edit, and use the model, but they must give you credit for the 
original work. 

¢ CC BY-NC—NonCommercial: Others cannot use your model 
commercially. 

¢ CC BY-ND—NoDerivatives: Others may use and share the mod- 
el, but it cannot be altered. 

¢ CC BY-SA—ShareAlike: Depending on the NC choices, others 
may share, edit, and use the model, but derivative work must be 
shared under the same license. (Sketchfab 2018a) 


Combined, the top ten most viewed 3D models in the category 
have been viewed more than 4,123,200 times. 

The popularity of Sketchfab within the cultural sector can be 
attributed largely to the fact that it is a free,° easy-to-use service 
with web browser and embed support. By supporting 3D annota- 
tions, audio, and animations, plus making it possible to view any 
uploaded model on the web in virtual reality (VR) and augmented 
reality (AR), the free Sketchfab app gives cultural organizations all 
they need to tell their stories using the objects and spaces in their 
care through 3D. 

It can be inferred from the trending data in figure 5-1 that the 
annotation functionality—the ability to add a clickable hotspot to 
a particular part of a 3D model—first piqued cultural organiza- 
tions’ interest in the Sketchfab platform. When clicked, the hotspot 
displays a text or image pop-up and transports the 3D viewer per- 
spective to a user-defined angle, zoom, and pan (figure 5-2). This 


3 https: / /iiif.io/community / groups /3d/#about 
“https: / /sketchfab.com/models/ categories / cultural-heritage-history 


5 Anyone can join Sketchfab on the free Basic tier (https: / /sketchfab.com/ plans); 
Sketchfab offers free Pro tier subscriptions and hosting to all cultural heritage 
organizations (https: / /sketchfab.com/ museums). 
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Fig. 5-1. Monthly sign-ups to the Museum segment on Sketchfab, 2013-2018 


feature enables the creation of virtual tours around a 3D model, with 
contextual and other information linked directly to 3D navigation. It 
may also suggest that clever 3D technology must be underpinned by 
a relevant use case that truly adds value in a given application. What 
use is a 3D model on its own? Only when the model is presented 
with contextual information that is strictly linked to the 3D nature of 
the media does it attract users from the cultural sector. 


SAVE VIEW 


Blow to the head? 


Fig. 5-2. The Jericho Skull—3D model by the British Museum on Sketchfab® showing 
an active annotation 


The ability to add annotations makes a 3D model more than the 
sum of its data; it creates a self-contained educational artifact, which 
is sometimes the main reason that organizations use Sketchfab to 


6 https: / / skfb.ly/RGHD 
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share 3D content online. For example, Jonathan R. Hendricks, direc- 
tor of publications at the Paleontological Research Institution and 
adjunct associate professor in the Department of Earth and Atmo- 
spheric Sciences at Cornell University, explains the significance of the 
Digital Atlas of Ancient Life program: 


The next step of our project is to annotate the existing models 
and incorporate them into both the textbook and into planned 
Virtual Teaching Collections, which will allow students access 
to organized, curated collections of virtual fossils when the 
real things are not available. (Jonathan R. Hendricks, personal 
correspondence, September 20, 2018) 


The second uptick in museum sign-ups in late 2015 can be attrib- 
uted to the launch of Sketchfab’s cultural heritage program, which 
offers free PRO tier subscriptions for all museums and cultural insti- 
tutions. This offer means that these organizations can use Sketchfab 
for free with the benefit of larger file uploads, more annotations, pri- 
vate models, and other advantages. 

A final increase in sign-ups from museums could also be at- 
tributed to the addition of support for webVR (a new web standard 
for browser-based VR content) in 2016, which allows users to click 
a button on any online Sketchfab model and jump into a VR mode, 
viewable on smartphone/ cardboard devices as well as on dedicated 
headsets. Although the long-term value of VR may still be under 
debate, the addition of webVR to Sketchfab may have interested au- 
diences beyond cultural organizations in the platform. This means 
that anyone producing 3D content could also become a producer of 
VR experiences, with no coding required. The same 3D model view- 
able on a regular screen can now be presented in a way that affords 
a level of presence (the feeling of visiting a place or space physically) 
and an impression of scale for the viewer. 

Certain functionality has not found widespread use. For exam- 
ple, the ability to add machine-readable tags to plot 3D models ona 
map’ would seem like a very practical feature, but it is seldom used 
at this point. Indeed, most institutions do not even tag their models 
on Sketchfab in a consistent manner or take advantage of Sketchfab’s 
APIs° to link 3D model data to their online collection databases to 
keep them connected and up-to-date. 

The increasing need for meaningful ways to display 3D models, 
in particular 3D models of historical and cultural content, is also con- 
firmed by the existence of several other online 3D viewers alongside 
the Sketchfab viewer, which was launched in 2012. These include the 
3D Hop (2015)? and the Universal Viewer (2012, 3D supported as of 
2017).'° Both offer open-source and cultural heritage-centric 3D view- 
ing experiences for those interested in self-hosting a 3D viewer. 

The emergence of dedicated 3D viewers from large commercial 


7 https: / /labs.sketchfab.com / experiments / map / 
8 https: / /sketchfab.com/ developers 

* http:/ /3dhop.net 

10 https: / / universalviewer.io 


77 


What Happens When You Share 3D Models Online (In 3D) 


companies, such as Google (Poly, 2017),'' Microsoft (Remix 3D, 
2016),'? and Facebook (3D Posts, 2017)'* suggests an even wider audi- 
ence for all kinds of 3D content. 


Audiences for Historical 3D Content 


By publicly sharing 3D data online for viewing, download, and re- 
use, we gain insight into the active audience for such historical 3D 
models. While researchers often acquire 3D data as part of academic 
study or for condition documentation during collection assessments, 
a much larger and more general audience is to be found online for 
cultural and historical 3D data. This suggestion arises from obser- 
vations of how users interact with cultural heritage 3D models on 
Sketchfab and how they share and reuse the same models both on- 
line and offline. 


Use by Publishing Organizations 

The simplest way to put 3D models online and show them to au- 
diences is to use a 3D viewer. Sometimes, as with the Harvard 
Semitic Museum," Minneapolis Museum of Art,’° and Chateau de 
Versailles,’° this means adding links or embedding 3D models from 
respective institutional profile accounts on Sketchfab in official col- 
lection pages and exhibition websites. The Royal Academy of Fine 
Arts (Madrid) and the Museo Archeologico Nazionale di Napoli 
employed a simple, but elegant, in-gallery solution, displaying web- 
based 3D models of original artifacts alongside physical plaster casts 
by using Internet-connected tablets (Marqués 2017). 

At other times, a third party uses 3D models published by one 
or more institutions. Sarah Bond, associate professor in the Classics 
Department at the University of lowa, uses 3D models from multiple 
Sketchfab users in her undergraduate and graduate classes, in place 
of physical replicas. She explains the relevance for her course: 


I have begun to integrate 3D models of inscriptions into my 
courses ... 3D models and digital humanities approaches to 
material culture provide ample opportunity for transporting 
students and the general public to “visit” and then translate 
inscriptions in situ. While nothing will ever replace doing 
squeezes and rubbings on-site, these are a close second when 
used in a browser, on a mobile device, or loaded into a VR 
viewer. (Bond 2018) 


Services like Sketchfab provide a framework for existing web 
teams to create sophisticated 3D interactive displays. For example, 


" https:/ /poly.google.com 

” https:/ / www.remix3d.com / discover?section=34b78f58881242e4ab611e4ab5 ffaa78 
3 https:/ / developers.facebook.com /docs/ sharing /3d-posts 

“https: / /semiticmuseum.fas.harvard.edu/3d-models 

% https:/ /collections.artsmia.org / art /3520/the-doryphoros-italy 

16 http: / / www.chateauversailles.fr / grands-formats / hameau-de-la-reine 
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the Natural History Museum in London used the Sketchfab Viewer 
API and models hosted on the platform to power a custom in-gallery 
touch screen for teaching visitors about whale skeletons (Shakiry and 
Capewell 2017). 


Outreach 

Placing 3D models online is an extension of the work that museums, 
archives, and libraries already do to make their location-specific col- 
lections accessible to a wider audience. Being able to measure this 
outreach in some way is even more valuable. Each model on Sketch- 
fab has some publicly visible statistics: the number of times the mod- 
el has been viewed anywhere online and the number of Likes and 
comments from members of the Sketchfab community. For example, 
at the time of this writing, the 3D scan Granite head of Amenemhat 
III’ published by the British Museum had been viewed more than 
111,000 times, had been downloaded 5,900 times, and had garnered 
667 Likes and 53 comments since its publication in 2014. It is notable 
that most of the Likes are from nonacademic users. To put the cul- 
tural heritage content on Sketchfab in context with the wider com- 
munity on the platform, consider that the 650 Sketchfab profiles in 
the Organization: Museum user segment (the label users give them- 
selves upon signing up to the service) represent less than 1 percent 
of the more than 2 million registered accounts on the platform. The 
total number of models in the cultural heritage category (uploaded 
by both institutions and individuals) accounts for about 2 percent of 
the total number of 3D models uploaded to Sketchfab. 

In addition to the direct views on Sketchfab, top embed referrals 
to the Granite head of Amenemhat III model include creativecommons. 
org; an educational resource site from Alabama Community College, 
.cgchannel.com (a computer graphics community site); and open- 
culture.com (an aggregator for publicly available online educational 
media). While potentially accessed by academics and museum staff, 
these sites are unlikely to be referred to as serious research portals. 
The thousands of viewers that the sites referred to the Granite head 
of Amenemhat III model on Sketchfab suggest a wider viewership for 
historical 3D data. 

Sketchfab is, as far as the author knows, the only community in 
which it is possible to leave a comment directly related to an individ- 
ual collection item—rendered in 3D or otherwise—from a museum, 
library, or archive. By commenting on 3D models, users show an 
interest beyond simply viewing items in a collection. Comments on 
the Granite head of Amenemhat III model range from simple thanks for 
the opportunity to view and download the 3D data to critiques of the 
data quality, questions about digitization workflow, and links to the 
data in new contexts. 

Sketchfab also allows any user to create their own groupings 
of 3D models from other users in Collections. Such collections have 
many purposes. They may be simple collections of favorite models, 


17 https: / /sketchfab.com / models /64d0b7662b59417986e9d693624de97a 
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collections created for specific uses (e.g., for use as artistic refer- 
ence!®), or collections intended to unite similar collections from dif- 
ferent institutions.” 

Interestingly, the Centre des Monuments Nationaux (CMN) in 
France does not publish 3D models on Sketchfab, but has created 
collections of 3D scans made by other users of monuments under its 
care.”” Although some institutions and estates are protective of their 
collection’s intellectual property, CMN wanted to encourage unoffi- 
cial crowd-sourced digitization efforts. According to Mélisande Vial- 
ard, Mission stratégie, prospective et numérique at CMN, 


At the moment, we don’t have a lot of models nor data because 
we are just starting to think about it. We would like to know 
more about how to work with the communities on Sketchfab 
especially because some of our monuments are already on your 
platform like the Sainte-Chapelle or the towers of La Rochelle 
and it would be great to encourage these initiatives. (Mélisande 
Vialard, personal communication, July 3, 2017) 


As 3D digitization tools become more accessible and simpler to 
use, many are taking up 3D scanning as a hobby and, increasingly, 


unofficial collections of historical 3D content are being posted online. 


As is the case with the models curated by CMN, hobbyist or com- 


mercial 3D scans fulfill a need where institutional capacity is lacking. 


There are, however, concerns about the diffusion of inaccurate or 
incomplete data. Daniel Pletinckx, the Cultural Technology Expert at 
Visual Dimension bvba suggests that 


Qualified objects could get a visible quality stamp. ... The 
important point here is to connect with the CH [cultural 
heritage] domain, so that CH experts start to feel connected and 
responsible for 3D CH resources. This would also stimulate 3D 
producers to deliver quality as the quality level of some items 
is questionable. (Daniel Pletinckx, personal communication, 
September 3, 2018) 


On occasion, Sketchfab has received and acted upon valid con- 
tent take-down requests, as occurred when the Artists Rights Society, 
representing Henry Moore, requested the removal of several 3D 
scans of the artist’s work from the platform. In this case, the copy- 
right claim is clearer than, for example, when referring to ancient 
monuments. However, Sketchfab is Digital Millennium Copyright 
Act (DMCA)-compliant and has a clear process for resolving any in- 
fringement claims (Sketchfab 2018b). 

In tandem with the decision to accept 3D models generated by 
the general public is the decision to release 3D data to the general 
public for download and reuse. Institutions publishing on Sketch- 
fab have the option to enable 3D models to be downloaded under 


18 https: / /sketchfab.com/demoon/ collections /ref_sculpture 
1 https: / /sketchfab.com/Tlatollotl/ collections / aztec 
0 https: / /sketchfab.com/leCMN / collections 
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several types of Creative Commons licenses listed earlier. Along with 
individual researchers and hobbyists, cultural organizations have 
made more than 10,000 cultural heritage—related 3D models available 
for download and reuse. 

The act of releasing 3D content under a generous license (i.e., 
permitting modification and reuse) means that cultural content can 
be reused in ways that are often unexpected. To take just a few ex- 
amples, the effect of the British Museum releasing many of its 3D 
scans for download led to a bust of Zeus hosting a YouTube show as 
a VR avatar (Smith 2018), 3D printed museums in a school classroom 
(Quince 2014), a hologram of Sir Robert Bruce Cotton (Interactive 
Studio 2017), and a livestream of a 3D print of a Pacific island god 
statue (Museotechniki 2016). 

In addition to the reuse of 3D scans for simple fun, there are 
examples of reuse for what might be considered more serious or 
worthy purposes. Archivist Abira Hussein has incorporated scans 
released by the British Museum in her work Healing Through Archives 
in multimedia webVR and tactile experiences,’ combining the 3D 
with 2D imagery, 360 panoramas, and audio interviews with the So- 
mali immigrant population in London. 

Some institutions, such as Réunion des musées nationaux 
(RMN)-Grand Palais” and the Grand Rapids Public Museum, have 
begun to license 3D data for commercial reuse on the Sketchfab 
Store.”? While the commodification of cultural data is not new,”* the 
monetization of a relatively new medium—and one that can be used 
to create convincing replicas of cultural objects—has prompted some 
comment. Ethan Gruber, director of data science at the American 
Numismatic Society, tweets his response to the sale of cultural 3D 
content: “Beyond the ethical implications for profiting from heritage 
acquired (or stolen) via colonialism, you may run aground of the law 
in various jurisdictions regarding indigenous art” (2018). 

The ethical question is bound up with the monetization of muse- 
ums in general and should be discussed along with such topics as in- 
stitutional funding streams, paid exhibitions, gift shops, and existing 
licensing models for other media and replicas. Regarding the legal 
question, Sketchfab (like most online media platforms) has channels 
in place for anyone to report rights violations and regularly acts on 
such reports to remove content and review licenses. Where there is 
lack of ethical or legal concerns, there is an opportunity for promot- 
ing the reuse of 3D content while creating a new revenue stream for 
cultural organizations. 


1 https:/ /sketchfab.com/models/7eb598e50b6f42d5a035eac152b6963e; https: / / 
twitter.com/nebulousflynn / status /880396725722152960 


2 https: / /sketchfab.com/francecollections/store 

3 https: / /sketchfab.com/grpm/ store 

4 For example, 2D image licenses from the Museum of Modern Art (http: / / www. 
scalarchives.com/web) and the Guggenheim Museum (http: / / www.artres.com) 
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Libraries and 3D 

Like museums, many libraries and archives are currently publish- 
ing 3D content. The British Library” (figure 5-3), National Library 

of Scotland” (figure 5-4), Cambridge Digital Library,” and State 
Library of Queensland” have all found that they have something in 
their print and special collections that can be experienced digitally in 
3D, and these are just the institutions on Sketchfab. 


SAVE VIEW 


Fig. 5-3. Jane Austen desk—Open view 2 by The British Library on Sketchfab?9 


Fig. 5-4. The Stag (stage set 1 of 5) by National Library of Scotland on Sketchfab?? 


5 https: / /sketchfab.com/britishlibrary 
26 https: / /sketchfab.com/natlibscot 

27 https: / /sketchfab.com/CamDigLib 
8 https: / /sketchfab.com/slq 

° https: / / skfb.ly /6xyxZ 

3° https: / /skfb.ly /6zF6G 
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@ save view 


Publishing content online in 3D opens up new and unique ways 
for the general public to interact with artifacts. Adi Keinan-Schoon- 
baert, digital curator (Polonsky Fellow) for the Hebrew Manuscripts 
Digitisation Project at the British Library, explains in a blog post: 


While a museum is more of a usual suspect for these novel 
technologies, libraries are perhaps less so. They are perceived to 
hold books, manuscripts, documents, or in short—compilations 
of two-dimensional text. But nothing physical that a library holds 
is in fact two-dimensional, and some items kept in libraries may 
be of unanticipated nature. Libraries have more potential to 
engage with 3D modelling and printing than one would expect. 
(Keinan-Schoonbaert 2016) 


Although publishing 3D data can be seen as part of libraries’ 
general goal to engage the public with their collections, it should also 
be considered (where appropriate) as a legitimate, yet underexplored 
way to fulfill research outreach targets. 

3D can augment or add value to 2D media, but the value is lim- 
ited by the kind of 2D or image artifact in question. Two examples 
illustrate the point. Figure 5-5 depicts a reuse of Miniature of Dunstan 
as a bishop, writing a commentary of the Rule of Saint Benedict, with an 
inscription “S[an]c[tu]s Dunstanus,” adapted from the British Library’s 
Catalogue of Illuminated Manuscripts. In this case, the additional 
material information has been added to the 2D data of the manu- 
script*! to allow a viewer to experience changes in the appearance of 
gold leaf on the manuscript as it is viewed from different angles. This 
could be said to be a more authentic experience of the manuscript 
compared with an on-screen or printed reproduction of the same. 


f..| 


YW hal 


1 & 


Fig. 5-5. Miniature of Dunstan as a bishop,?? adaptation by Thomas Flynn 


3t http: / /www.bl.uk/ catalogues /illuminatedmanuscripts /ILLUMIN. 
ASP?Size=mid&IIIID=52658 


2 https: / /skfb.ly /69FL6 
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Figure 5-6 illustrates the digital morphing of a map from 1888 
with more modern elevation data added and contour lines overlaid. 
It is possible then to see the difference in the way that a particular 
area of land has been documented over time and the possible devia- 
tion in survey accuracy. Thus, figure 5-5 illustrates an artistic addi- 
tion to augment the user experience, while figure 5-6 shows how the 
addition of scientific data can enable the user to view a historic docu- 
ment in a new way. 


@ snve view 


‘ erarsdenes . \ ee 2 ee? som ae 


Fig. 5-6. Planta da Cidade de Ouro Preto - MG - 1888,°% adaptation by Rolling Drone 
Geotecnologias 


Conclusion 


The potential use cases for 3D data have not yet been entirely ex- 
plored. By putting 3D models online and making use of existing 
platforms and functionality, however, we are starting to get a sense 
of current possibilities and limitations, and to learn what audiences 
react to and enjoy. The biggest benefit of putting 3D content online 
may be that it increases the visibility of, and in turn engagement 
with, 3D data. 

As examples of the use and reuse of 3D data are circulated, their 
value and usefulness will be challenged and contested. While the 
simple “technical magic” of 3D digitization can impress and excite 
now, the novelty will wear off and the focus will move to the ques- 
tion of how 3D can help achieve existing institutional goals. 

In the last few years, many people who are producing or work- 
ing with 3D data in the cultural heritage sector have moved from 
“wow!” to “why?” as they assume a more critical view of the value 
of 3D capture and display. As individual researchers; 3D community 
groups; museum, archive, and library staff; and those working in in- 
ternational think tanks take up the task of establishing standards and 


33 https: / /skfb.ly /6BIqY 
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recording project outcomes, we move toward a time when 3D data 
will play as ubiquitous a role as imagery, video, and text currently do 
in our everyday work. 
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Chapter 6 


Building for Tomorrow: 


Collaborative Development of Sustainable Infrastructure 
for Architectural and Design Documentation 


Ann Baird Whiteside 


Abstract 


The use of 2D and 3D computer-aided design (CAD) and building 
information modeling software is now routine in architecture and 
design firms and in design education programs. CAD is particularly 
problematic for the libraries, museums, and archives responsible for 
the long-term management of design documentation as CAD is high- 
ly volatile, relying on proprietary mathematical algorithms to repre- 
sent shapes and structures, and is packaged in complex, proprietary, 
and rapidly evolving software products that are expensive, digitally 
encrypted, and obsolete within years. Architectural museums and 
archives are facing a rapidly growing need to preserve digital infor- 
mation and are grappling with the need for technological tools, tech- 
nical expertise in digital preservation, AutoCAD expertise, archival 
expertise, and repositories that can preserve and disseminate the ar- 
chived data. Many institutions, especially smaller ones, lack the tech- 
nical infrastructure and expertise to implement scalable preservation 
of design records. A community-based approach to building an 
infrastructure comprising technology, digital preservation strategies, 
standards, and education for archivists of these collections is critical 
if we are to preserve digital architectural records at scale. In 2017, the 
Frances Loeb Library at the Harvard University Graduate School of 
Design applied for a grant from the Institute of Museum and Library 
Services, Building for Tomorrow, to support the convening of a na- 
tional forum under the National Digital Platform funding priority 
during 2018. The grant supported two priority-setting meetings of 
engaged stakeholders to frame a national / international collaborative 
infrastructure to support the long-term preservation of digital design 
data, specifically in the architecture and design fields. 
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Background 


he preservation and management of architecture and design 

records have undergone significant change over the last 40 

years as architects, engineers, and other designers move to the 
use of digital technology for design documentation. Archivists and 
librarians responsible for keeping this documentation have long been 
concerned about how it will be preserved. In architecture and design 
projects, many types of digital files are produced during planning and 
construction, and these files are important for long-term preservation 
for future renovations / restorations and scholarly research. Comput- 
er-aided design (CAD) is particularly problematic for the libraries, 
museums, and archives responsible for the long-term management of 
design documentation because CAD is highly volatile, relying on pro- 
prietary mathematical algorithms to represent shapes and structures, 
and packaged in complex, proprietary, and rapidly evolving software 
products that are expensive, digitally encrypted, and obsolete within 
a few years. Libraries and archives are increasingly under pressure to 
acquire these twenty-first century collections to support the next gen- 
eration of architectural students and historians. 

Since the introduction of CAD software in the 1960s, industries 
that design and develop our built environment have been moving 
from pencil and paper to computers and digital files. The earliest 
adopters of the new technology were the aerospace and automotive 
industries; they were followed enthusiastically by the fields of ar- 
chitecture and design. CAD allows architects to take previously un- 
imaginable risks in their designs and to experiment with new forms 
and materials without building prototypes or performing expensive 
structural analyses until much later in the process. U.S. architects 
such as Frank Gehry led the way, and schools of architecture have 
been teaching the technical expertise needed to unite architecture, 
engineering, and software design. This has led to a new generation 
of architects who leverage technology and has opened new doors for 
innovation in design. 

The use of 2D and 3D CAD and building information modeling 
(BIM) software is now routine in architecture and design firms. De- 
signers typically use multiple types of CAD software throughout the 
design process, which is characterized by phases: concept, schematic 
design, design development, construction, and as-built. For many 
architects and firms, a building project now comprises tens of thou- 
sands of digital files that include 2D drawings; 3D models; video; 
images; and communications among architects, clients, contractors 
and other parties, including e-mail messages, contracts, specifica- 
tions, requests for information, and architects’ supplemental instruc- 
tions. 3D models may consist of multiple interrelated files, requiring 
deep knowledge to understand, that are usually held by the project 
architect. In addition to 3D CAD models, there are hundreds or even 
thousands of detailed 2D-layer drawings produced for particular as- 
pects of a building; there are 3D printed objects, and there are project 
“outputs”—for example, drawings or sketches of the building. There 


Building for Tomorrow: Collaborative Development of Sustainable Infrastructure for Architectural and Design Documentation 


are photographs and videos of the construction site, websites about 
the building, BIMs, and other multimedia related to the project. 
With CAD, architects and designers can modify files throughout the 
design lifecycle, allowing for a less linear and more flexible design 
process and more innovative design. Although the architecture pro- 
fession has standards for managing design files, firms often use con- 
ventions that are easiest to use within the firm and may not be easily 
translatable to those outside the firm. 

Several challenges are associated with the software in the archi- 
tecture profession. There are multiple programs used throughout a 
design project; the software has highly robust functionality, which 
also makes it difficult to preserve; and the software is proprietary, 
making migration of the data to other software systems difficult. The 
software relies on complex mathematical algorithms to represent 
shapes and structures. Furthermore, software products change rapid- 
ly, and they are expensive, encrypted, and quickly become obsolete. 
Alex Ball, research officer at UKOLN in the United Kingdom and at- 
tached to the Digital Curation Centre (DCC) based at the University 
of Bath, described the issues in 2013 in this way: 


The issue of poor interoperability between CAD systems 

and between versions is exacerbated by the rate of software 
development. In order to maintain a competitive edge, there is 
constant pressure on CAD vendors to release new versions of 
their software with increased functionality or fewer limitations. 
Not only does this create instability regarding file formats and 
their interpretation, it also means that individual versions of 
CAD packages can become obsolete rather quickly (Ball 2013, 10). 


To preserve the records of significant building projects complete- 
ly, all the digital information should be captured and linked or pack- 
aged together into a collection that can be easily searched and, once 
found, navigated and preserved over time. 

In the analog and early digital world, the contractual deliverable 
to clients was a set of printed, wet-signed and wet-stamped drawing 
sets. We have moved to a model of electronically signed 2D and 3D 
files that can be manipulated to convey information that is at least as 
detailed as traditional printed plans. Further, over the last five years, 
students in architecture and design schools have been routinely us- 
ing CAD for modeling, skipping the 2D drawing process entirely, 
meaning that the coming generation of architects will be producing 
documentation only in 3D models, adding urgency to the problem of 
preserving this type of documentation. 

The impact of the shift to an entirely digital architectural pro- 
duction workflow on the record of architectural innovation and 
practice—in architecture libraries, archives, and museums—is only 
beginning to be understood. No longer can libraries acquire blue- 
prints or drawings, a few images, and a scale model or two to repre- 
sent a major work of architecture in their collections. Now they must 
acquire the 3D CAD models and 2D drawing files, BIMs, digital im- 
ages, videos, and documents, delivered on a computer hard drive, 
often with no annotation whatsoever. 
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Fortunately, the standards for CAD are advancing so that options 
are emerging to represent CAD drawings and models in ways that 
achieve a degree of interoperability across systems and time. These 
standards are complex and offer many trade-offs. Such standards 
include ISO 14641:2018 Electronic Document Management — Design 
and Operation of an Information System for the Preservation of Electronic 
Documents—Specifications; ISO 20614:2017 Information and Documenta- 
tion — Data Exchange Protocol for Interoperability and Preservation; Indus- 
try Foundation Classes (buildingSMART International 2018); Standard 
for Exchange of Product Model Data ISO 10303 (STEP Tools, Inc. 2018); 
and Initial Graphics Exchange Specification (IGES 2018). 

Different software programs support different standards, and 
each standard supports different aspects of the represented design. 
Archiving these digital design files raises a host of questions about 
the purposes that the digital designs should serve, their authenticity, 
and the best ways to manage such assets technically in the digital 
future. 

To complicate matters further, multiple CAD software vendors 
supply the architecture, engineering, and construction industry, 
including Autodesk, Inc., Bentley Systems (Microstation), Dassault 
Systémes (CATIA), and others. As with all digital software, file for- 
mat obsolescence is a barrier to our ability to archive many digital 
design files. Architectural museums and archives are faced with a 
rapidly growing need to preserve digital information and are grap- 
pling with the needs for technological tools, for technical expertise in 
digital preservation, for AutoCAD expertise, for archival expertise, 
and for repositories that can preserve and disseminate the archived 
data. Over the last two decades, the risk of losing this portion of our 
digital cultural heritage has grown tremendously. 


History of Previous Work (2003-2017) 


Over the last 15 years, architectural practitioners, archivists, and 
technologists have been working on the problem of preserving digi- 
tal design work. The Art Institute of Chicago initiated the first project 
in the United States; it lasted from 2003 to 2005 and was conducted 
by Kristine A. Fallon, an architect. The research project of the Mas- 
sachusetts Institute of Technology (MIT), Future-proofing Architec- 
tural Computer-Aided Design (FACADE), took place from 2007 to 
2009 and was conducted by technologists, librarians, and architects. 
The Society of American Archivists Design Records Section has been 
doing related research and surveys in the archival community since 
2012. The 2014 International Confederation of Architecture Museums 
(ICAM) conference addressed these issues in several presentations. 
In November 2017, the Library of Congress, the Architect of the 
Capitol, and the National Gallery of Art hosted a summit, Designing 
the Future Landscape: Digital Architecture, Design and Engineer- 
ing Assets, at the Library of Congress. Finally, the Canadian Centre 
for Architecture has led the effort for the profession, using tools and 
techniques from the digital preservation community. 
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Art Institute of Chicago 

In 2003, the curatorial Department of Architecture at the Art Institute 
of Chicago undertook a study to identify requirements for creating 
and maintaining an archive of born-digital objects, with architect 
Kristine A. Fallon as principal investigator. 

The first step of the study was to conduct a survey to determine 
how design firms were then using digital design tools, what types 
of digital design data were being produced, how important digital 
design data were to understanding the design process, and whether 
digital methods were the primary way that the firms did work. If 
digital methodologies were not already primary in their work, re- 
spondents were asked whether they thought that digital methodolo- 
gies would be primary within five years. They were also asked to list 
the specific software products that they used in each use category. 
Survey respondents reported using digital design tools in all catego- 
ries identified in the survey. The most frequently cited were in the ar- 
eas of communication/ presentation (94 percent) and documentation 
(93 percent). The least frequently cited use was for rapid prototyping 
(24 percent). More than half of the respondents reported using digital 
working methods primarily in data gathering, documentation, com- 
munication/ presentation, and design exploration (Kristine Fallon 
Associates 2004). 

The second step was to conduct in-depth case studies of projects 
ranging in scale from industrial design to urban design at nine U.S. 
design firms. The case studies showed that digital design tools are 
integral to the design process and digital images are central to design 
decision-making. 

The third step was to validate that the findings drawn from the 
case studies could be generalized to the broader design community. 
This was done through an international survey, in which staffs at 
design firms were asked how they used digital design tools, how 
important the tools were to their practices, and which products they 
used. 

The team also conducted research into earlier archiving projects 
and existing standards, methodologies, and products for collecting 
and archiving digital design data. It was found that no museum or 
archival institution had solved the key problem of ensuring long- 
term preservation of the numerous and rapidly changing data for- 
mats from digital architecture projects. 

Basing their assessment on the Open Archival Information Sys- 
tem (OAIS) reference model for a long-term data repository system 
(ISO 14721:2002), the team identified six distinct stages of the work- 
flow (planning / programming, design, construction, closeout /com- 
missions, operations / maintenance, and disposal) for bringing digital 
design data from design office to museum or archive and for making 
it accessible to the public. 

The report included recommendations on procedures, technol- 
ogy and related requirements, and a start-up implementation plan. A 
summary of the recommendations includes the following guidelines: 
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¢ The design practitioner should organize, name, and maintain de- 
sign data in such a way that a curator or archivist can discern the 
contents of data files and the time sequence in which they were 
produced. The designers should preserve important outputs— 
drawings, images, and animations presented to clients—in archi- 
val formats. 

¢ Once the archive and a design firm have defined the content of 
a gift of digital design data, the design firm should prepare the 
Submission Information Package (SIP). SIPs should contain the 
content files and some level of descriptive information, including 
file-naming standards or project directory structure for a given set 
of files, and should be sent to the archive for ingest. 

¢ Recommended archival formats suitable for inclusion in the SIP 
include the following: 
¢ PDF/A and TIFF 
¢ MPEG2 
e PNG 
¢ BMP 
e Extensible 3D 
e Universal 3D 


The Art Institute of Chicago’s report provides thorough recom- 
mendations and requirements for the implementation of a complete 
preservation plan for digital design records. Recommendations are 
included for institutional funding, hardware and software infrastruc- 
ture, digital storage, data maintenance, cataloging and access, and 
personnel. The recommendations distribute the workload of digital 
preservation between creators and archivists, and assume firms and 
collecting institutions will have the technical infrastructure and tech- 
nology expertise required. 


MIT FACADE Project 

In 2007, the MIT Libraries received an Institute of Museum and Li- 
brary Services (IMLS) grant to develop a strategy for processing and 
preserving the digital outputs of architectural projects involving 2D, 
3D, and other digital files. The FACADE project team worked with 
the offices of Frank Gehry, Moshe Safdie, and Thom Mayne to create 
a collection of digital material from these major architects. The collec- 
tion was to include different 3D CAD modeling tools that the archi- 
tects had used in their normal work practices and that could be used 
as a research test bed. The project team was provided with design 
files for specific projects for the research. 

The research explored the best way to associate 3D designs with 
related 2D drawings, digital images and videos, e-mail messages 
and other communications, and BIMs. Specifically, what techniques 
should be applied to preserve native CAD models over archival 
timeframes? Is it necessary to preserve software, or is an emulation 
framework required? What additional process information is re- 
quired to capture the building lifecycle, and how can that be stored 
in digital archives? What other annotations must be supported to 
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retain the architect’s intentions and instructions to contractors and 
subcontractors who do the construction, and how should this in- 
formation be kept? How do we archive this information into digital 
preservation repositories and make it accessible? 

Five major areas of investigation were defined for the project: 
(1) analysis, identification, and description of major CAD formats; 
(2) analysis, design, and implementation of native CAD file inges- 
tion; (3) management, preservation, and dissemination practices; (4) 
analysis and recommendations related to process documentation 
(relationships among various CAD files and versions, and between 
CAD files and other documentation); and (5) training, outreach, and 
dissemination of results to the digital library and digital preservation 
communities. 

Material was acquired on a hard drive or set of DVDs, in the file 
system in use by the firm, and without annotation to help determine 
what was included. Of the test collections acquired, the size ranged 
from just under 20,000 files (10 GB) to almost 100,000 files (50 GB) for 
a building in progress. The 3D CAD models in particular were each 
very large (comprising one or more separate files), but usually few 
in number. The 2D CAD drawings and other files were smaller, but 
numerous. Each firm responded to requests for project files differ- 
ently, depending on their own practices. As happens in the analog 
world when collections are acquired, some firms simply turned over 
all the files; others had already culled the project files for their own 
archives, in which case the team acquired a smaller set consisting of 
what the firm considered important to keep. Ideally, the team would 
acquire complete sets of data that included not just the designs and 
client presentations, but other archival material that is often of high 
historical value. 

For each building, the team sought material from all stages of the 
project, including concept design, schematic design, design devel- 
opment, construction documents, and construction administration. 
While 3D models were the focus of the research on digital preserva- 
tion, the context provided by the other materials in the collection 
(e.g., client presentations, correspondence with clients and contrac- 
tors, and digital images) was key to understanding the models. Un- 
like analog architectural drawings, which show through line draw- 
ings the design intent of an architect, CAD drawings and models are 
intricate and multilayered, with layers combined to produce one file. 
The intent of the architect’s design is integrated into the layers of the 
files and into the references between files. As a result, the combina- 
tion of files, rather than a single file, is necessary to understand the 
architect’s design intent. 

The project team developed a set of recommended best practices 
to support a preservation strategy that covered all the materials re- 
ceived in a building collection. These recommendations included 
e Special processing of 3D CAD models to generate derivative ver- 

sions with greater long-term archiving potential than the native 
software format. The team identified the need for four versions 
with distinct formats to ensure long-term preservation: 
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1. Original (the originally submitted version of the CAD model) 

2. Display (an easily viewable format to present to users, nor- 
mally 3D PDF) 

3. Standard (full representation in preservable standard format, 
normally IFC or STEP) 

4. Dessicated (simple geometry in a preservable standard format, 
normally IGES) 

¢ Semi-automated conversion processing of other key design file 
formats (e.g., 2D drawings into PDF files). 

e Automated conversion processing of common digital file formats 
(e.g., Microsoft Office documents and JPEG images) as part of ar- 
chive ingest. 

¢ No special processing for the remaining classes of file formats, al- 
though these will come under more generalized digital repository 
preservation strategies outside the scope of FACADE’s focused 
concerns. 


For the project, the team was able to acquire all the CAD soft- 
ware products used by the architects who contributed to the research 
collection, and the team had valid access to those products through- 
out the project. Should an archive need to keep CAD software in 
perpetuity to view older CAD models, the archive would have to 
continue buying license keys for the software forever and hope that 
those CAD companies do not go out of business. This is obviously 
not a realistic strategy for long-term preservation; ideally, access 
to software will be maintained for many decades. The team dis- 
cussed this issue with representatives of several of the leading CAD 
software companies, and they were open to the idea of escrowing 
unrestricted copies of the software with appropriate libraries and ar- 
chives. The team concluded that this was the best avenue to pursue. 

The team performed a detailed case study of emulation as a 
strategy on the AccuDraw software on the Apple II platform (long 
since obsolete), and the team was able to view AccuDraw models by 
running the software in a simulated environment. The process and 
lessons learned were documented in detail, and the team felt that 
this approach was technically viable for preserving modern CAD 
software and data. However, the issue of legal access to the software 
via license keys remains a significant barrier. 

Community discussions made it clear that many institutions, es- 
pecially smaller ones, lack the technical infrastructure and expertise 
to implement the preservation models developed by the FACADE 
team and the Art Institute of Chicago. A key take-away was that a 
more community-based approach to building an infrastructure with 
technology, digital preservation strategies, standards, and education 
for archivists of these collections is critical if we are to preserve digi- 
tal architectural records at a national or international scale. 

Following the FACADE project, MIT and the Harvard Graduate 
School of Design did further work in 2011 and 2012. This included 
developing a more robust set of metadata fields for the descrip- 
tion of digital design files and continuing work on a prototype of 
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the original accessioning tool built in the first FACADE project. The 
work was funded by a small grant from the Harvard Library and did 
not continue because of a lack of funding and changes in personnel 
at each institution. 


Society of American Archivists Design Records Section 

In 2013, the Society of American Archivists Design Records Section 
launched a CAD/BIM Task Force as a catalyst for a community-wide 
initiative to address the numerous legal, technical, and curatorial is- 
sues of born-digital architectural records. The work of the task force 
to date includes a survey of firms and architectural archives that was 
used to learn about holdings and current archival practices for born- 
digital design data, the development of a born-digital studies bib- 
liography, and a report of the task force (Leventhal and Zalduendo 
2013, 2014). 


ICAM17 Conference 

In September 2014, ICAM held its conference in Montreal and New 
York.' The opening session was devoted to the topic of archiving 
born-digital architectural materials. 

Representatives from five international institutions of varying 
sizes described efforts at the institutional level to deal with digital 
design data. They discussed such issues as collecting strategies, soft- 
ware curation, and tools for archiving and description. The overall 
impression from across the presentations was that the complex prob- 
lems inherent in preserving digital design data are multilayered and 
comprise many general challenges. Each institution was developing 
its own processes and methodologies, and many were not working 
across allied domains, such as digital preservation, from which they 
could adapt solutions and standards. The presentations made it clear 
that the problems would be better worked on collaboratively across 
institutions and across domains to take advantage of expertise at 
both the national and the international levels. 


Library of Congress 

In November 2017, the Library of Congress, the Architect of the 
Capitol, and the National Gallery of Art hosted a summit, titled 
Designing the Future Landscape: Digital Architecture, Design and 
Engineering Assets, at the Library of Congress (Leventhal 2018). 

The summit brought together 180 stakeholders in the architecture, 
design, and engineering professions, from creators to curators, to 
explore the issues and obstacles of long-term preservation and access 
to the records of their projects and to begin working toward sustain- 
able solutions. The critical issues that arose through the summit were 
the need to identify the full array of digital design files created; the 
need to determine which design records, and specific types of data 
or information in those records, the various stakeholders need in 

the immediate and long-term future; and the need to develop better 


http: / / www.icam-web.org /data/media/cms_binary / original /1405105617.pdf 
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communication and information-sharing practices, which are critical 
to developing sustainable solutions to problems with the long-term 
preservation, access, and use of digital design files. 

Having so many representatives from the range of digital design 
communities engaged in discussion at the summit indicated how 
pressing the issues of preservation are, as we risk losing vital cultural 
heritage information. Discussions highlighted the fact that advance- 
ments in digital preservation tools, such as BitCurator and Archive- 
matica, and collection management tools such as ArchivesSpace, of- 
fer technological support for the preservation of digital design files. 
The communities at the summit also recognized the importance of 
working with colleagues across domains to support the preservation 
efforts. 


Building for Tomorrow 

The research that has been done from various perspectives since 2003 
has paved a path for understanding the digital preservation needs 
for 3D architecture and design files. We have tested different pres- 
ervation methods and tools, and we have begun to engage the com- 
munities—from those working in creation through those working 

in preservation and access. To further this work, the Frances Loeb 
Library at the Harvard University Graduate School of Design (GSD) 
applied for a grant from IMLS? in 2017 to support the convening of a 
National Forum under the National Digital Platform funding priori- 
ty during 2018. The grant supported two priority-setting meetings of 
engaged stakeholders—architects, architectural historians, archivists, 
librarians, technologists, digital preservationists, and others—who 
are framing a national /international collaborative infrastructure to 
support the long-term preservation of digital design data, specifi- 
cally in the architecture and design fields. The infrastructure includes 
the ongoing integration of knowledge, standards, technologies, and 
management across generations of technology and practice. The first 
meeting, a day-and-a-half-long forum, was held immediately prior 
to the Society of Architectural Historians annual conference in St. 
Paul, Minnesota, in April 2018. The outcomes of the forum will be 
published in spring 2019 along with a set of strategic directions and 
actions forming the basis for a strategic plan of community-based 
work in the area of preservation for digital design records. 

At asecond meeting, a steering committee met May 30-31, 2018, 
at the Harvard University GSD. The goals of the meeting were to 
refine the group’s strategic directions and actions, and to determine 
its immediate next steps. A significant area of focus in the plan is the 
development of connections to other communities that have intersec- 
tions with this work, including the software industry, those working 
in the realm of 3D and VR, and those who provide digital preserva- 
tion tools and technologies, in order to leverage expertise and find 
commonalities across disciplinary domains. 


2 https:/ / www.imls.gov/sites/default/ files / grants /1g-73-17-0004-17 / proposals /lg- 
73-17-0004-17-full-proposal-documents.pdf 
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An outcome of the May steering committee meeting was a deci- 
sion to build a formal coalition of stakeholders. A proposal for the 
structure of a Building for Tomorrow coalition of allied communities 
is being drafted and will be shared for community comment as part 
of the engagement strategy. Participants in Building for Tomorrow 
will engage design records community members who could not be 
present at the forum, specifically software vendors with whom we 
want to engage, and will convene a meeting of those working on 
three current IMLS projects focused on 3D preservation (Building for 
Tomorrow, Community Standards for 3D Preservation [CS3DP], and 
Developing Library Strategy for 3D Virtual Reality Collection Devel- 
opment and Reuse [LIB3DVR]). We will also convene another meet- 
ing of the steering committee, which we hope to use as a jumping-off 
point to formalize a Digital Architecture, Design, and Engineering 
(DADE) coalition. 


Conclusion 


Over the last year, as a result of the community work in this area and 
through the Building for Tomorrow meetings, several steps have 
been taken to help move this work forward in a community-based 
model. We have the tools, technologies, and technological expertise 
to preserve digital architecture, design, and engineering data, and we 
can apply them to this domain. Several efforts in the United States 
focus on developing standards for preservation for 3D data, and we 
need to collaborate across our communities to leverage the expertise 
across those domains. We need to garner the engagement of design- 
ers in practice to incorporate thinking about preservation at a far 
earlier stage in their careers. We need to push harder to engage the 
software industry in order to align incentives for preserving software 
with the need to preserve the files created using the software. We 
need to think about sets of services around the preservation of digital 
design files to meet the needs of institutions of various sizes. And, to 
be successful, the preservation of digital design files must be an ef- 
fort across domains and institutions. 
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Chapter 7 


3D/VR Preservation: 


Drawing on a Common Agenda for Collective Impact 


Jessica Meyerson 


Abstract 


Digital curation is now part of the repertoire of all computationally 
dependent domains, requiring mechanisms for alignment to address 
common digital curation challenges that are beyond the scope of 
any individual organization or domain. This chapter explores the 
collective impact methodology (CI) and its application to cultural 
stewardship and digital curation challenges within and beyond the 
3D and virtual reality (VR) discourse. To demonstrate the relevance 
of CI to 3D/ VR preservation, the chapter provides a CI case study on 
software preservation that is experiencing early success in aligning 
efforts across sectors toward the long-term preservation and reuse 
of software. The chapter concludes by posing questions for consid- 
eration by 3D/VR practitioners as well as possible next steps to ad- 
dress one or more 3D/VR data curation challenges. 


Introduction 


na 2017 article, John Wenzler argues that one of the greatest chal- 

lenges to the realization of a sustainable scholarly communica- 

tions infrastructure is the collective action dilemma—the inherent 
difficulty in looking beyond localized, organizational needs toward 
“conscious coordination’ in the management of the scholarly re- 
cord” (196). Wenzler’s article inspired focused responses, including 
one regarding the development of financial mechanisms to sustain 
open scholarly infrastructure (Lewis 2017) and another advocat- 
ing the use of social alignment mechanisms to direct the focus of 
individual institutions toward common goals (Neylon 2018). The 
collective action dilemma is characterized by Wenzler, Lewis, and 
Neylon as the primary challenge to the sustainability of an open 
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scholarly infrastructure. It is also a challenge faced by cultural stew- 
ards engaging in digital curation across the information management 
landscape. 

Digital curation is now part of the repertoire of all computation- 
ally dependent domains, necessitating alignment mechanisms to ad- 
dress common digital curation challenges that are beyond the scope 
of any individual organization or domain (Digital Curation Centre 
2018; Poole 2016). 3D and virtual reality (VR) data preservation! is 
treated here as both a digital curation use case—positioned at the 
intersection of some of the most difficult digital curation challenges, 
including scale, standards and interoperability, and hardware and 
software dependencies—and a boundary-spanning area requiring 
alignment of interests across a broad range of stakeholders, includ- 
ing the software industry, hardware manufacturers, academia, cul- 
tural stewardship organizations, standards bodies, policymakers, 
publishers, and (re)users of 3D/ VR collections. 


Collective Impact 


As an alternative to the isolated impact of single organizations acting 
alone, collective impact (CI) is a collective action approach to com- 
plex social challenges that accounts for conflicting institutional or 
domain-specific priorities by design (Kania and Kramer 2011). 

Kania and Kramer (2011) argue that successful Cl initiatives 
share these five conditions: 

1. Ashared vision, or common agenda that engages all of the stake- 
holder communities” 

2. Ashared system of measurement that serves to measure the ef- 
fectiveness of varied stakeholder activities and outcomes toward 
advancing the common agenda 

3. Mutually reinforcing activities that leverage the unique contri- 
butions of each stakeholder group in a way that complements 
and amplifies the work of all stakeholder groups 

4. Constant communication to demonstrate a commitment to fair- 
ness and evidence-based decision-making over time 

5. A backbone organization with paid staff to plan, coordinate, and 
sustain the effort through facilitation, technological infrastruc- 
ture, and data collection, synthesis, and reporting 


Tn this chapter, 3D/VR preservation refers to (1) preservation of the means 
(parameters, data, and software) to render the volume and surface of a 3D object 
mathematically and (2) preservation of the “technological system(s) that use 
interdependent hardware and software to place users in a computer-generated, three- 
dimensional environment that is immersive and interactive” (Campbell 2017, 7). 

? Infrastructure studies make a useful frame for expanding the potential scope of 
stakeholder communities for software. Specifically, in their in-depth study of the 
WORM online community, Star and Ruhleder (1996) outline several key characteristics 
of infrastructure that apply to software, including embeddedness, transparency, reach, 
and scope. They also note that such infrastructure should embody standards and be 
learned as part of a membership, be built on an installed base, be fixed in modular 
increments, and be visible on breakdown. 
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The greatest concentration of CI case studies is found in the 
fields of education, public health, homelessness, youth development, 
urban and economic development, community development, and 
the environment (CCHD 2018). One of the longest running well-doc- 
umented applications of Cl is Strive Together, “a national, nonprofit 
network of 70 community partnerships” working “to ensure that 
every child succeeds from cradle to career, regardless of race, income 
or zip code.”* Strive Together represents 12 years of experimentation, 
testing, and refinement of their “theory of action,” which serves as an 
invaluable resource for understanding the evolution of CI over time 
in response to shifting social and economic circumstances. Through 
this and other examples, John Kania and Mark Kramer (2013) have 
argued that CI emphasizes “complexity and emergence” as two 
characteristics of system-level challenges that CI is particularly well 
suited to address.* 


Telescoping the 3D/VR Literature—Common 
Data Curation Challenges for Complex Data 


3D/VR is a collective action problem, evidenced through a specific 
set of challenges that individual institutions or individuals cannot 
tackle including scale, standards and interoperability, and hardware 
and software dependence. We need a collective action solution to ad- 
dress these challenges if we want to progress past highly constrained 
and localized 3D / VR curation environments to ensure 3D/ VR pres- 
ervation over the long term. 

According to Lischer-Katz, the challenge of archiving VR objects 
arises from the “complexity of [these] data objects, in terms of the 
variety of data types, relations between files, and dependencies on 
a variety of hardware and software platforms” (2017, 6). While 3D/ 
VR is associated with disciplinary, format, and platform-specific 
challenges, many of the digital curation challenges cited in the 3D/ 
VR literature are common for a broad range of complex digital data. 
These challenges include the implications of scale for stewardship 
and access, standards and interoperability, and dependency on spe- 
cific hardware and software environments for meaningful access. By 
positioning 3D/VR preservation at the nexus of these common data 
curation challenges, we identify areas where collective action can 
make the greatest impact on long-term preservation. 


3 https: / / www.strivetogether.org / about / 

“Tn “Collaborating for Equity and Justice: Moving Beyond Collective Impact,” Tom 
Wolff and coauthors summarize the limitations of CI, including the “failure to cite 
advocacy and systems change as core strategies, engage those most affected in the 
community as partners with equal power, and directly address the causes of social 
problems and their political, racial, and economic contexts.” The authors propose 
six principles for collaborative practice that promote equity and justice. These six 
principles inform a broader framing of CI discussed throughout the text of this 
chapter (Wolff et al. 2017). 
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Scale 

The challenge of scale has many facets in 3D/ VR preservation, in- 
cluding file size, volume of files, number of possible platforms and 
formats, and the geometric complexity of the environment or object 
that is represented by a 3D/VR rendering. While geometric complex- 
ity is unique to 3D/VR preservation, the volume, size, and range of 
potential formats to be curated or preserved for a single VR environ- 
ment represent well documented digital curation challenges. For 
example, in the field of digital archives, digital corpora for a single 
user’s storage media can easily come to tens of thousands of files 
(Sloyan 2016). In the context of distributed research project teams, 
the interdisciplinary Zuse Institute Berlin (ZIB) points out that “the 
trend in large-scale research data management constitutes a para- 
digm shift from previous local data storage in file systems and data- 
bases to the coordinated orchestration of unstructured, high-volume 
data maintained in distributed sites.”° 


Standards and Interoperability 

In a digital curation ecosystem in which numerous tools are applied 
to a growing volume of digital assets at each phase of the workflow, 
standards and interoperability are key. The tools themselves fre- 
quently change, and granular metadata facilitates the reuse of out- 
puts from one tool to another. 

The business case for standardization and interoperability are 
based on two well-grounded assumptions: (1) standards facilitate 
interoperability by allowing the exchange of information across con- 
texts, and (2) interoperability facilitates flexibility in the approach to 
data curation because the interpretation of the outputs of curation 
activities is not dependent on a given tool. For example, audiovisual 
preservation professionals have been particularly concerned with 
the persistence of embedded metadata due to varying levels of sup- 
port for the metadata across the range of software applications used 
to render audiovisual files (Lacinak and Forsberg 2011). Similarly, in 
3D /VR-based practice, “digital watermarking” describes a family of 
techniques that embed uniquely identifiable information about an 
information object in the object itself to address “security aspects of 
data and user authentication or data integrity protection” (Steinbach, 
Dittman, and Neuhold 2008). Digital watermarking makes it pos- 
sible both to detect unauthorized uses of the intellectual property 
(e.g., the underlying 3D models) and to track the chain of custody 
through distributed technological infrastructures. Tools to support 
access to working 3D representations and reproducibility of those 
representations would have to be able to track specific instances of a 
model across systems. However, this level of interoperability relies 
on standardization around the specific properties and, in some cases, 
the machine-readable encoding standard used to represent these 
metadata. 


5 http:/ / www.zib.de/ research /large-scale-data-management-curation-analysis 
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According to the principal investigator and the co-investigators 
of the Community Standards for 3D Data Preservation (CS3DP) 
project, there are currently no agreed upon standards for the preser- 
vation of 3D data (Moore et al. 2018). More specifically, establishing 
agreed upon schemas for provenanical metadata remains a chal- 
lenge (Cohen-Boulakia et al. 2017). An agreed upon provenancial 
schema would facilitate interoperability among systems and tools 
included in the workflow, as well as interoperability of execution 
environments for reproducibility or replay. PROVdata is a schema 
adopted by some virtual heritage practitioners; however, equivalent 
standards do not exist in every domain that employs 3D / VR tech- 
nologies (Koller, Frischer, and Humphreys 2010). One can hope for 
a common set of extensible elements that could permit searches for 
complex digital objects across domains (e.g., all models within the 
same coordinate system) and within a single domain (e.g., all mod- 
els of a particular monument that have a low uncertainty quotient), 
similar to the implementation of Preservation Metadata Maintenance 
Activity (PREMIS) in digital repository and preservation systems to 
track presentation “events” over the life of a digital object.® 

As community infrastructures grow to include distributed 
networks of shared software and data across participating 
organizations,’ it is crucial not only to track provenance and pres- 
ervation events associated with assets across their life, but also to 
make that metadata discoverable. Such a system feature enables 
organizational and individual users to know the source location 
and to use the provenancial chain of events associated with the 
lifecycle of a digital object as a second-order data set. 


Software and Hardware Dependence 
According to the authors of “Research Challenges for Digital Ar- 
chives of 3D Cultural Heritage Models”: 


Anecdotal evidence suggests that digital models that are only 
two or three years old are losing their original functionality 

and information richness because of poor archival practice. 

CAD software products used to create virtual heritage models 
belong to a rapidly changing market segment, which means that 
updated versions are inevitable (on the order of every 18 months) 
and the full reuse of data (without loss of information) is not 
secured at all. (Koller, Frischer, and Humphreys 2010, 7:14)8 


Likewise, findings from the Pooling Activities, Resources and 
Tools for Heritage E-research Networking, Optimization and Syner- 
gies (PARTHENOS) project include an existing and “important gap 
between open-source solutions . .. and the opaque alternatives of 
commercial black boxes . . . raising the question of our software de- 
pendency” (Alliez et al. 2017). 


6 https: / / www.loc.gov/standards / premis / 
’ For example, Software Heritage, Datacite Re3Data, EaaSI. 
* For an in-depth study of the longevity of computer-aided design, refer to Smith 2008. 
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Software and hardware dependencies have been similarly dis- 
cussed in other data curation contexts, including time-based media 
conservation (Laurenson 2006), archives and special collections 
(Meyerson et al. 2017), and peer review in light of an expanding 
definition of a well-formed research object (Bailleul 2016, 42). Of the 
common 3D/VR digital curation challenges listed earlier, software 
dependence arguably has the broadest reach. All sectors and dis- 
ciplinary domains use some form of software as part of their work 
process. While the function and user community for a specific soft- 
ware title varies widely, some software dependency challenges span 
domains. 

The transition from a shared acknowledgment of the need for col- 
laboration to a sustained effort aimed at system-level change is one 
of the key social infrastructure challenges that CI is meant to address. 
The American Library Association (2014), the Educopia Institute (Lip- 
pincott and Skinner 2014; Skinner, Drummond, and Halbert 2014; 
Skinner and McCain 2016), and the Institute of Museum and Library 
Services (2017) are among those who have embraced CI as a concep- 
tual framework within cultural stewardship. Of these, the cultural 
stewardship CI case studies with the greatest extensibility to 3D/VR 
preservation have been undertaken by Educopia Institute, which has 
translated its strategies to collective action challenges that are topical 
rather than regional in scope. The Software Preservation Network 
(SPN) is the most recent affiliated community of Educopia, and is ac- 
tively applying the CI framework to “Saving Software, Together.” 


Software Preservation Network: A Case 
Study in Collective Impact 


Before detailing SPN’s application of Cl it is critical to outline the 
recurring challenges found in the software preservation discourse 
(Meyerson, Potterbusch, and Work 2018). Layering major events in 
digital preservation over the software preservation timeline high- 
lights broader trends that have directly impacted the creation, pres- 
ervation, and reuse of software and digital data, including changes 
to U.S. copyright law that favor copyright holders (Aufderheide et 
al. 2018); the period of annual price reduction for computational 
resources (Koller, Frischer, and Humphreys 2010); the growth of 
the institutional repository landscape (Lynch and Lippincott 2005); 
increasing acquisition of hybrid collections (Redwine et al. 2013); 
funder requirements for data sharing and reproducibility of compu- 
tationally dependent research (National Institutes of Health 2018; 
National Science Foundation 2018); and a shift away from migration 
and toward emulation as a long-term digital access strategy (Berg- 
meyer 2011; bwFLA 2018; Granger 2000; Rosenthal 2015; Rothenberg 
1999; Waters and Garrett 1996). 

Whether your data is a simulation of climate change impact on 
ocean sea levels or a CAD file from 2000 or Deus Ex, there are four 
major software preservation challenges that recur in the discourse 
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over time, are applicable to any domain, and must be addressed 
through programmatic and strategic collaboration. These challenges 
are as follows: 


Challenge 1: No single institution can feasibly locate, much less 
acquire or procure, all the software needed to address their existing 
born-digital data. 


Implication: Coordinating collection development and sharing of 
software resources among cultural stewardship organizations is es- 
sential for long-term preservation. 


Challenge 2: Intellectual property regimes in different nations are 
conflicting. 

Implication: Leveraging existing legal tools across national con- 
texts (and sharing a body of anecdotal evidence from researchers 
and practitioners, which is crucial to advocating for the expansion 
of stewardship and user rights) is necessary to ensure that intellec- 
tual property law does not restrict the preservation of and access to 
software. 


Challenge 3: As distribution models change from physical instal- 
lation media to software as a service, software libraries and ex- 
ecutables are squarely and exclusively in the control of software 
producers. 


Implication: Articulating and aligning the shared needs / interests of 
cultural stewardship organizations in order to represent community 
needs and capabilities in conversation with the software industry is 
essential to ensure that software producers do not dictate preserva- 
tion decisions. 


Challenge 4: There are no broadly adopted standards for describing 
the technical, provenancial, and relational properties of software. 


Implication: To make software discoverable and preservable, it is 
essential to analyze the body of existing descriptive metadata stan- 
dards for software and identify their common properties. Once a 
content-level ISO standard has been developed, creators and practi- 
tioners can use it to map their localized metadata implementation to 
the broader software preservation ecosystem. 


In response to these challenges, the Software Preservation Net- 
work builds on existing collections and digital preservation infra- 
structure to support the specific preservation and access needs of 
software. 


Common Agenda 

A common agenda is a shared vision or goal for change that en- 
gages all of the stakeholder communities around a problem area. 
Historically, software preservation efforts have focused largely on 
technological developments or on attempts to tackle the full range of 


3D/VR Preservation: Drawing on a Common Agenda for Collective Impact 106 


challenges within a single domain. SPN exists as a commons for sec- 
tors to align and prioritize resource investment around shared chal- 
lenges. This in no way obviates the need for more domain-specific 
initiatives, tools, working groups, and organizations. Instead, we 
want to help amplify that work and funnel some of that collective 
energy toward international-scale challenges to effectively promote 
cross-domain discussions that have representation from all of those 
areas. The common agenda that unites SPN’s participants and sup- 
porters is to preserve software through community engagement, 
infrastructure support, and knowledge generation. SPN tackles this 
agenda across five core activity areas: law & policy, metadata & stan- 
dards, training & education, research-in-practice, and technological 
infrastructure.” Communication, governance, and advisory groups 
support the work of the five activity areas. 


Shared Measurement 

The SPN Research-in-Practice Working Group is focused on improv- 
ing shared measurement through review and synthesis of previous 
software preservation (and related) research. The purpose of this 
synthesis is to create a Software Preservation Research Toolkit con- 
sisting of (1) a centralized longitudinal data effort to track progress 
and developments in the software preservation landscape, and (2) a 
set of data-gathering instruments that individuals can use to gather 
data about software preservation and curation in their local organi- 
zation or community (Hagenmaier 2017). By aggregating findings 
from across contexts and survey instruments, practitioners in the 
field can more easily map the landscape of software preservation and 
curation, allowing us to focus energy and expertise more precisely 
and effectively. 

Shared measurement—“collecting data and measuring results 
consistently on a short list of indicators at the community level and 
across all participating organizations” (Kania & Kramer 2011)—is 
one of the most difficult and also one of the most powerful compo- 
nents of collective impact. One key insight resulting from analysis 
of the software preservation discourse is that the only measurable 
change over time is an increase in the number of project-based and 
programmatic efforts addressing some aspect of software preserva- 
tion. This measure does not, however, advance understanding of 
whether, how, and to what extent these efforts have enabled pres- 
ervation, sharing, and long-term reuse of software. Without shared 
measurement, there is no empirically-grounded way for stakeholders 
to determine whether past and current efforts have successfully ad- 
vanced the state of the field. 


Mutually Reinforcing Activities 

In addition to requesting that the SPN working group coordina- 
tors map their action plan items to strategic goals, SPN requested 
that the working group coordinators map their activities to the 


° http: / /www.softwarepreservationnetwork.org / about / 
Pp Pp 8 
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related activities of other working groups. A shared understanding 
of unique stakeholder contributions and roles allows each group 
to play to their strengths and interests while highlighting the value 
of participation to all groups. In a series of SPN all-hands strategy 
meetings held in January and February of 2018, network members 
articulated specific mutually reinforcing activities shared among 
working groups. Through the documentation of those activities, 
members reported increased trust in the process as whole. 

SPN identifies affiliated projects in addition to mutually rein- 
forcing working group activities.'’ SPN’s involvement in an affili- 
ated project is first and foremost by tracking the project goals and 
measures of success against the collective agenda. Beyond shared 
measurement, SPN may shape an affiliated project or serve in an as- 
needed advisory capacity. Currently, SPN affiliated projects include 
Scaling Emulation as a Service Infrastructure (EaaSI)," Fostering 
Communities of Practice for Software Preservation in Libraries, Ar- 
chives and Museums (FCoP),!? and Code of Best Practices for Fair 
Use in Software Preservation.’ 


Constant Communication 

As mentioned earlier, meetings have been a crucial component of 
building SPN. The SPN Forum in 2016 resulted in the development 
of a community roadmap that drives the first SPN working groups 
and has been used as the foundation for iterative development of the 
current SPN mission, vision, values, and strategic goals. The 2018 
SPN all-hands strategy meetings allowed all the current members 

of the network to clarify the relationships between group goals and 
broader strategic goals, provide real-time feedback about priorities, 
and identify gaps and opportunities. Between larger convenings 

of network members, monthly working group meetings, monthly 
working group coordinator meetings, quarterly advisory committee 
meetings, and bimonthly newsletter and e-mail communications, 
SPN stays in regular communication with both active participants 
and the SPN community at large. The minutes of all meetings are in 
a shared Google Drive, and meetings are always expected to result in 
time-bounded, distributed tasks that address the group’s action plan. 
Ad hoc meetings also take place between different groups or sets of 
stakeholders. 


Backbone Organization Support 

SPN has undertaken, with some success, the coordination and man- 
agement of the day-to-day facilitation work, including stakeholder 
engagement, communications, data collection and analysis, and 
other responsibilities. However, the last two years have shown that 
relying solely on dedicated volunteers for organizationally critical 
tasks can present challenges. Organizations have paid support roles 


10 http: / / www.softwarepreservationnetwork.org/ projects / 

1 http: / / www.softwarepreservationnetwork.org/eaasi/ 

? http: / / www.softwarepreservationnetwork.org/fcop/ 

3 http: / / www.softwarepreservationnetwork.org /bp-fair-use / 
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(such as communications specialist, financial manager, systems ad- 
ministrator, program manager) because certain functions are ongoing 
and critical for both the survival and the accountability of an orga- 
nization. SPN, Community Standards for 3D Preservation (CS3DP), 
and Building for Tomorrow (B4T)"" are awarded grants with some 
staff hours and some travel support for external participants. While 
this level of funding may be sufficient for a two-year period in which 
volunteer participants work together to articulate the priorities for 
action, dedicated full-time staff are eventually essential to push the 
common agenda forward and track activities to the system of shared 
measurement. 

In August 2018, SPN launched a membership and sponsorship 
campaign and a two-year seed funded project with the support of 
19 members and counting.’* Membership and sponsorship are open 
to organizations in every sector committed to the preservation and 
long-term reuse of software. While the launch of the campaign is a 
direct response to the challenges of sustaining Cl initiatives without 
dedicated resources, the formalization of governance and adminis- 
trative operations will, by necessity, affect the dedicated community 
of individuals and organizations that have driven the effort thus far. 
3D/VR practitioners should consider SPN and other documented CI 
efforts as a source of information that could improve their efficacy 
for 3D/VR preservation. 


Conclusion: Collective Impact for 
3D/VR Preservation 


There are numerous points of intersection between the software pres- 

ervation agenda, general data curation goals for complex data sets, 

and 3D/VR curation challenges. Listed below are several questions 
for the 3D/ VR community of practice (Wenger 1999) to consider; these 
questions may inform collective action for the curation challenges 
unique to 3D and VR data: 

¢ Because 3D/VR preservation encompasses numerous digital cura- 
tion challenges, should 3D/VR preservation be framed as a collec- 
tive impact problem? Or is it potentially more effective to focus on 
each major data curation challenge that bears particular relevance 
for 3D/VR? 

¢ Where are the discussions about community infrastructure taking 
place? Who is leading them? Is there currently a backbone organi- 
zation that can drive the alignment efforts necessary to sustain a 
Cl effort for 3D/VR or associated digital curation challenges? 

e Which 3D/VR stakeholders are missing from the current 3D/VR 
preservation discourse, and how might the problems be framed 
differently to ensure that those stakeholders become part of a 
sustained and evolving solution to the inherent data curation 
challenges? 


“4 https:/ / projects.iq-harvard.edu/buildingtomorrow /home 
5 http: / / www.softwarepreservationnetwork.org / prospectus / 
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e What forms of shared measurement make sense for a Cl effort aimed 


at 3D/ VR preservation? 
What are the major gaps or issues with the CI model? What are 
the blind spots, particularly for tackling 3D/VR preservation? 


The following are recommended next steps toward a CI ap- 


proach to 3D/VR curation challenges: 


Identify past 3D/ VR project and programmatic goals and associ- 
ated outcomes in multiple domains. 

Determine clear gaps in the landscape and use them to set a com- 
mon agenda. 

Make the components of the agenda measurable, and explicitly 
identify key stakeholders that are well positioned to lead work 
related to specific components of the agenda. 

Publish the agenda for feedback, and elicit participation from 
stakeholder groups currently unrepresented in the 3D/VR preser- 
vation discourse. 

Track current and future project and programmatic goals and as- 
sociated outcomes to the common agenda. 

Determine which existing organizations have the capacity to serve 
as the backbone organization for a 3D/VR collective impact initia- 
tive, and if this model is unsuitable, determine the most appropri- 
ate model to support that work. 


Looking ahead, the enterprise of digital curation for 3D /VR data 


depends on the presence of actors at every level of participation; 
acknowledgment of the unique contributions of individuals, orga- 
nizations, and domains; and a CI approach that enables the 3D/VR 


community to direct those contributions toward a common agenda. 


While the complexity of 3D/VR data raises many hurdles, most of 
the essential curation challenges can be addressed in concert with 
work being done in the broader digital curation community. 
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CS3DP: Developing Agreement for 3D Standards 
and Practices Based on Community Needs and Values 


Jennifer Moore, Adam Rountrey, and Hannah Scates Kettler 


Abstract 


The pace and growth of 3D model creation has increased tremen- 
dously in recent years, a trend that will continue as technology 
evolves. The ever-present need to develop practices and standards to 
ensure the long life of this data type has never been more apparent to 
both 3D creators and data curators. In this moment, a community is 
coming together in an effort to increase the accessibility, discoverabil- 
ity, usefulness, and integrity of data. One such effort, the Community 
Standards for 3D Data Preservation, is focused on collective develop- 
ment of flexible and extensible best practices and/or standards for 
preservation, documentation, and dissemination of 3D data. 


The Growth of 3D Data Creation and the 
Need for Curation 


echnical advancements in creation and capture of 3D data 
| and a reduction in costs have inspired intense growth and 

interest in creating, sharing, and using digital 3D data in the 
last decade. The digital 3D medium is still rapidly evolving; new us- 
ers and creators are coming in from many backgrounds, often with 
different end goals. Yet, as users and creators, we are also at a point 
in the development and usage of 3D data at which many recognize 
the need for a system of standards—or at least guidance—for the 
documentation, preservation, and dissemination of 3D data. Locally 
developed standards are now starting to emerge independently in 
labs, museums, libraries, and businesses, often with little communi- 
cation among stakeholders. This could be problematic for the larger 
community, as a system of standards optimized in isolation solely for 
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the needs of a specific field or discipline is unlikely to meet the needs 
of those in other fields and may even be completely unusable outside 
its home field. For example, a standard designed specifically for 3D 
models of archaeological sites and collecting events is unlikely to 
meet the needs of game developers and architects, and it may unnec- 
essarily limit the discoverability of the content. 

Collaboratively developed standards would increase accessibil- 
ity, discoverability, and usefulness outside the field of the user or 
creator, and they would foster innovative applications and scholarly 
work by reducing the barriers to cross-disciplinary access. How- 
ever, the more generalized standards require compromises, which 
means that they may not fully meet the needs of any specific field. 
Given the need for documentation and preservation standards in 
most fields using 3D data, the broader community is now choosing 
to come together at a powerful, formative time, asking questions 
not only about how information should be encoded for sharing and 
preservation, but also about what we value and how we can design 
a system that promotes those values. In other words, what kinds of 
things do we want our standards to encourage and facilitate? 


Relationship of Data Curation and 
3D Data Today 


Historically, librarians, curators, and museum managers have been 
concerned with curation and preservation of scholarly works and 
physical objects. In SPEC Kit 354 produced for the Association of Re- 
search Libraries (ARL), the authors reported that two-thirds of ARL 
libraries surveyed were engaged in or developing data curation ser- 
vices in their libraries at some level (Hudson-Vitale et al. 2017). Over 
the last decade, digital data curation services and research on digital 
preservation have become part of the established mission of many 
libraries, but few have explored the complexities of 3D data curation 
in depth. 

At first glance, 3D data may appear to be “just like any other 
data,” but this is not a full characterization. Like digital photographs 
or simulation results, 3D models are, in the most reduced sense, just 
bits and bytes. This means that some parts of the preservation pro- 
cess, such as file storage and fixity, are not substantially different for 
different data types. Of course, there is more to preservation than 
just storage. One could argue that the goal of preservation is to main- 
tain an object, ensuring that it may be “used” (even if in a restricted 
way) over a longer period. For a 2D image repository to be useful, 
potential users need to be able to judge the suitability of an image for 
an intended purpose. They may need information about the photog- 
rapher, location, date, subject, and licensing. In some cases, detailed 
information about the photography equipment and post-processing 
may be necessary. A 3D repository has a similar preservation mis- 
sion—to make 3D data usable over a long period—but the informa- 
tion required to determine whether the data are fit for a particular 
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purpose is different. Because of the growth, complexity, lack of 
standardized documentation, and rapid change in 3D data formats 
and creation methods, the larger 3D community is only beginning to 
assess what is needed to ensure that 3D datasets remain sufficiently 
useful in order to consider them preserved. 

It is important that more data curators and librarians are now 
exploring the problem of 3D data curation, but agreement among 
librarians and curators is not enough to ensure its preservation; stan- 
dards for preservation are useful only if adopted by the creator and 
user communities. 


Aligning Creators and Curators 


The 3D/ virtual reality (VR) ecosystem includes a broad array of 
communities and stakeholders that are involved in the creation, 
preservation, and dissemination of 3D data. Libraries, museums, and 
archives should engage with this large community to assess needs 

in 3D data curation and agree upon practices and standards. Indeed, 
the first step for the Community Standards for 3D Data Preservation 
(CS3DP) project, funded by the Institute of Museum and Library 
Services (IMLS),' was to develop a better understanding of stake- 
holders’ needs in the 3D data creation and data curation community 
(Moore et al. 2017). 

We distributed a survey to libraries, museums, university labs, 
and government agencies, and asked their staffs to share it with 
other interested parties. In our initial survey, we received more than 
100 responses from individuals who worked with 3D data or had re- 
sponsibility for curating data. Notably, 72 percent of all respondents 
said that they were not using documented best practices or standards 
for preservation, documentation, and dissemination of 3D data. Of 
this group, 69 percent said that they did not use them because they 
were unaware of such standards. The respondents who were using 
standards had largely developed them in-house. The survey data 
made it clear that we were at a critical point in standardization. Ex- 
isting standards lacked buy-in or were so poorly known that local 
ad hoc standards were being created to fill in the gap. The increase 
in local standards was not indicative of a desire for independence or 
isolation in preservation practices, as the majority (85 percent) of re- 
spondents said they would like to collaboratively develop standards 
and best practices as a community (Moore and Scates Kettler 2018). 
Community members clearly recognized the barriers to collabora- 
tion and aggregation introduced by local solutions and were ready to 
work together on a more unified system of standards. 

The CS3DP project emerged from the idea that the adoption of 
standards depends not only on meeting the needs of the community, 
but also on community ownership and stakeholder investment. We 
used the survey results to build a five-part framework for organizing 
ideas: (1) preservation best practices, (2) management and storage, 
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(3) metadata, (4) copyright and ownership, and (5) access and dis- 
coverability. These framework topics eventually led to working 
groups that focused on considering the details of each topic. Howev- 
er, unanticipated discussions during the planning and subsequent fo- 
rum may have been some of the most important. For example, at the 
first gathering of CS3DP participants, many remarked that the wide 
variety of fields and backgrounds represented at the gathering was 
both unusual and much needed. In fact, the breadth of backgrounds 
and expertise was so diverse that we made it a priority to develop 
agreed-upon vocabularies for use in our discussions so that we could 
speak to one another with better clarity and understanding. Exper- 
tise was sufficiently diverse that we needed standards to even start 
to talk about standards! Questions that arose in discussions showed 
the desire of the community to work together and develop a general 
understanding of the current state of 3D preservation. Among the 
questions raised were “What is 3D data?” “What is it that should be 
preserved?” “How do we facilitate sharing without discouraging cre- 
ativity and innovation?” and “What stakeholders are missing from 
these discussions?” 

The next step in the process of developing 3D standards is in- 
cluding perspectives that were not adequately represented in the ini- 
tial survey, including, but not limited to, those of indigenous and na- 
tive communities, those from nations with technological bandwidths 
different from those commonly found in North America and Europe, 
and those from the entertainment industry. Targeted engagement 
with these communities in the form of actual relationships, not just 
consultations, is appropriate if the future of 3D data preservation is 
to take into account the actual breadth of the needs and requirements 
of all the users and creators of 3D content. 

This concept was brought to the fore as Narcisse Mbunzama 
spoke as part of the first CS3DP forum on February 6, 2018. During 
the panel on discovery and access (Wittenberg, Nieves, and Mbunza- 
ma 2018), Mbunzama discussed the innovations in 3D in the Demo- 
cratic Republic of the Congo (DRC) and use cases there that heavily 
rely on mobile devices for dissemination and access. The 3D data 
are shared via the DRC’s unreliable cellular and internet services, 
and are processed using basic computers. Yet, the contribution and 
scholarship surrounding 3D data is no less active in the DRC than it 
is anywhere else in the world. A standard and set of best practices for 
3D preservation should account for the concerns of such active com- 
munities and should meet the needs and requirements of various 
types of users. 


Bridging Efforts in the Community 


The process of collaboratively building 3D data curation has only 
just begun in earnest. However, our efforts are building on and 
incorporating stakeholders from well-established projects such as 
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Morphosource,? Smithsonian 3D,? Cultural Heritage Imaging,’ and 
FACADE (Smith 2008), as well as initiatives such as the Guides to 
Good Practice® from the Archaeological Data Service and Digital An- 
tiquity and the 3D Icons Report on Metadata and Thesaurii (D’ Andrea 
and Fernie 2018), funded by the European Commission and other 
European-based funders. These efforts have been extremely valuable 
in setting the tone for discussions. Yet, they tend to be focused on 
particular disciplinary needs or specific 3D data creation methodolo- 
gies. Because of this focus, the applicability beyond specific contexts 
of data creation is limited. We want to assess the previous work and 
build on it to develop 3D data curation practices and standards that 
will be broadly applicable and extensible to meet the needs of multi- 
ple use cases and user groups. Such a set of standards would support 
the creation of simplified, broadly applicable preservation systems as 
well as enhanced accessibility through data aggregators. 

The CS3DP project is not alone in tackling the aforementioned 
concerns and is building connections with other projects. These 
bridges will facilitate communication about diverse needs and con- 
cerns and will promote the development of standards that will be 
used by a larger interdisciplinary community. The CS3DP team has 
tracked preceding efforts, and we continue to monitor current efforts 
to address the curation of 3D data, including the National Endow- 
ment for the Humanities (NEH)-funded forums on 3D in 2015 and 
2016, which focused on the scholarly rigor, research output, and user 
experience rather than the preservation of 3D content (NEH 2018); 

a white paper resulting from the PARTHENOS Workshop held in 
France in late 2016 (PARTHENOS 2016); IMLS-funded 2018 forums, 
including Developing Library Strategy for 3D and Virtual Reality 
Collection Development and Reuse (LIB3DVR) (Virginia Tech 2018); 
Building for Tomorrow (Harvard University 2018); the International 
Image Interoperability Framework (IIIF) Community 3D Interest 
Group (IIIF 2018); and the CLIR 3D/VR colloquium (3D/VR Cre- 
ation and Curation in Higher Education 2018). These groups’ efforts, 
the cross-pollination of our teams, and the conversation that is con- 
tinually growing more focused have proven invaluable in identifying 
problems to be addressed in 3D preservation, community standards 
development, and the formation of best practices. 

In sharing our experience and utilizing our larger collective 
network, can we begin to deepen our understanding of each other’s 
needs, current practices, and shortcomings for 3D data preservation? 
In the short term, the CS3DP team anticipates that the contributors to 
these projects will maintain cross-pollination and collective growth, 
and will move forward together to support widely applicable 3D 
data standards and recommendations. 


2 https:/ / www.morphosource.org / 

3 https: / /3d.si.edu /about 

*http:/ /culturalheritageimaging.org / 

5 http:/ / guides.archaeologydataservice.ac.uk/g2gp/3d_Toc 
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Extensible Standards That Encourage 
Innovation Through Congruity 


In an ideal 3D /VR ecosystem, 3D data standards and practices facili- 
tate curation, but do not inhibit innovation, extensibility, and adapt- 
ability. The CS3DP team sees this as an imperative for any 3D stan- 
dard or best practice. As articulated during multiple 3D practitioner 
gatherings (Humanities Heritage 3D Visualization 2015, LIB3DVR, 
CS3DP, Building for Tomorrow, and CLIR’s 3DVR colloquium), a 
way forward is for the community to build a minimum set of stan- 
dards and guides that include mechanisms for review, amendment, 
and a la carte extension. Although somewhat aspirational, we believe 
this can be accomplished by focusing on identifying the commonali- 
ties among disciplines, modalities, and use cases. 

As we come to understand our common needs, we can build 
congruity through practical tools. Templates for documenting 3D 
data workflows that are both modality-specific and unobtrusive 
could provide a structure that fosters efficient dissemination, col- 
laboration, and assessment of 3D data. Agreement on documentation 
could permit the evaluation of a 3D resource for specific tasks and 
contexts and could uphold and expose scholarly rigor. The commu- 
nity is working now to identify needs, extant methods, and vocabu- 
laries and to provide direction on how this documentation can be 
created and linked. 

An analysis of ongoing requirements is taking place, which aims 
to make it possible for repositories to be optimized for long-term 3D 
data preservation and for access. Congruous 3D data repository re- 
quirements will support the discovery of 3D assets that are years old, 
allowing future scholars to reuse, re-create, and remix these data. 
Repositories will be able to describe which data holdings have been 
treated for preservation and persistence, which will in turn inform 
reusers about the data set’s integrity. That sort of quality assurance 
permits datasets to be retrieved, studied, augmented, and cited to 
advance research or allow for the production of new scholarship. 
Through community consensus we will move toward interoperabil- 
ity by establishing preferred formats that will make vendor adoption 
practical. In multiple forums we have heard arguments for respon- 
sive platform design that can deliver heavy data for use in powerful, 
high-performance computers, as well as decimated data for use on 
small devices, such as a smartphone, which is particularly important 
for countries that depend on mobile accessibility. 

We do not envision there being “one repository to rule them all,” 
but the community may at some point be able to build a tool that ag- 
gregates metadata following the standards on which we agree. An 
aggregator would allow access from disparate repositories so that 
users could search across disciplines to find digital 3D assets. For 
example, an animator for the Discovery Channel could search for a 
3D rendering of a plesiosaur and find a reconstruction offered from 
the University of Michigan with appropriate metadata that would 
enable them to determine whether the 3D asset would be useful, 
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appropriate, and accurate, without duplicating effort. Users should 
be able to download, derive, augment, and redistribute an object 
with its provenance attached as embedded metadata or, at the very 
least, clearly articulated and explained in related documentation. 

Through a common understanding of what is central to 3D data, 
a community-developed approach to data curation will facilitate re- 
producible workflows and provide methods by which to evaluate the 
integrity of data objects. Established requirements will allow creators 
to reference guidelines that not only will support data management 
planning, but also will inform workflows and documentation of the 
data generated and the arguments created. Preservation of these data 
is essential in supporting the basis of scholarship. Curators value 
openness and sharing, but that must be balanced with cultural sensi- 
tivity, a rights structure that supports creator innovation, and proper 
recognition for novel forms of scholarship. Such a structure will also 
provide a basis for the measurement of 3D scholarship and thereby 
help to facilitate promotion and tenure based on these data. 

Determining rights and attributions for creators and scholars 
is not the only consideration while developing common needs for 
3D preservation. Understanding 3D data creation and ownership 
is problematic, as suggested by Angel Nieves in his talk at CS3DP 
Forum 1 (Wittenberg, Nieves, and Mbunzama 2018), because cul- 
tural heritage assets have been largely in the hands of the privileged. 
Adopting a lens grounded in diversity, inclusion, and equity by 
including additional voices of marginalized user and creator com- 
munities will ideally address some of the biases within digital collec- 
tions that reflect a singular, sometimes predatory, cultural perspec- 
tive. By integrating our communities’ diverse needs and perspectives 
early in the establishment of a standard 3D data curation process and 
throughout its development, we can begin to build real relationships 
that address the prevalence of bias and violence done by the acad- 
emy, archives, libraries, and museums. A best practice or standard 
steeped in the values of diversity, inclusion, and equity can promote 
social justice through the inclusive curation and preservation of 3D 
cultural resources. 

Once the community of diverse 3D scholars and contributors 
has been established, we may yet succeed in creating an inclusive, 
collaborative 3D data preservation ecosystem. Community collabora- 
tion is necessary to build flexible standards and practices that afford 
librarians and data curators the tools that they need to support the 
3D community’s continued innovation, development, open dissemi- 
nation, and further discovery of 3D research. With the appropriate 
consideration of different perspectives, we will generate the neces- 
sary fervor to support 3D research where we can share, use, find, and 
benefit from each other’s data well into the future. 
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being used for their scholarly and pedagogical benefits. Sup- 

porting these technologies furthers the mission of academic 
libraries to ensure that their constituencies have access to scholarly 
information in all forms and formats. The potential benefits of 3D/ 
VR technology can be fully realized only when the technology is 
properly integrated into research programs and teaching curricula, 
an area in which the library can lead. Yet, the essays also make it 
clear that there remains a range of critical considerations that library 
professionals need to keep in mind as they shepherd novel 3D/VR 
technologies into their institutions, especially as they find themselves 
supporting new and complex technical workflows, scholarly prac- 
tices, and data curation and digital preservation requirements. 

The great diversity in the range of stakeholders involved compli- 
cates the development of comprehensive technical tools. One of the 
benefits of the CLIR 3D/VR colloquium was that it not only brought 
together a diverse range of stakeholder groups and enabled knowl- 
edge sharing across often-siloed groups, but also helped to identify 
stakeholder groups that the planning committee had not identified 
before the 3D /VR discussion. 

The box on the following page shows the stakeholder groups 
represented at the colloquium, including invited experts and other 
attendees from the broader University of Oklahoma academic com- 
munity, as well as additional stakeholder groups that had not been 
included in the colloquium. 


Ts essays in this report offer a glimpse at how 3D and VR are 
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Diverse Stakeholders 


The following stakeholder groups were represented at 
the CLIR 3D/VR colloquium: 
e 3D technologies and architecture specialists 
e Archaeological 3D data researchers 
e Independent 3D animators 
e Industry 
— Corporate 3D scanning practitioners 
— 3D model sharing platform representatives 
e Library communities 
— Academic library administrators 
— Digital humanities librarians 
— Digital library applications developers 
— Digital preservation specialists 
— Digital scholarship specialists 
— Emerging technology librarians 
— Entrepreneurship librarians 
— Fine and applied arts librarians 
— GIS and anthropology librarians 
— Librarians of reference and instructional services 
— Postdoctoral fellows in data curation 
— Social sciences and humanities librarians 
e Meteorologists 
e Nonprofit preservation organization representatives 
from the Software Preservation Network and CLIR 
e University faculty members with expertise in 
— Advanced medical imaging 


The following groups, not represented at the collo- 
quium, were also identified as stakeholders: 
e Architecture libraries, archives, museums 
e Industries 
— Aerospace 
— Automotive 
— Games and entertainment 
— Software development 
e Museum patrons 
— Children with disabilities 
— Elementary school educators and students 
e Museum staff 
e Public library patrons 
e Public library staff 
e Students, undergraduate and graduate, in 
— Architectural history 
— Art 
— Computer science 
— Digital humanities 
e Underrepresented communities 
— Communities with different technological bandwidths 
(i.e., impact of “digital divide”) 
— Elderly populations 
— Indigenous and native communities 
— Minority communities seeking to preserve their 
cultural heritage 


— Biology e Other university programs/schools 
— English — Law school libraries 

— Fine arts, sculpture — Schools of architecture 

— Journalism 


— Media arts and sciences faculty 


These expansive lists illustrate the wide range of communi- 
ties that have an interest in the development of standards and best 
practices around 3D /VR technologies. No longer does this area of 
research interest only a small group of specialists. From schoolchil- 
dren to advanced researchers across multiple disciplines, the use and 
impact of 3D/VR are expanding rapidly and becoming increasingly 
mainstream. 

In part because of this growing range of stakeholder groups, 
there is still much debate about who will lead the way in establish- 
ing the 3D/VR ecosystem identified in this report. For example, will 
researchers and other content creators, libraries and archives, or 
commercial platforms take the lead? Should the ecosystem be a cen- 
tralized one or a more loosely associated network? It is clear that the 
key elements of any 3D/VR ecosystem should include the following 
tools and modules: 
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e Auniversal viewer that integrates with existing and emergent 3D 
creation workflows and VR visualization tools. 

¢ Tools and metadata schemata that enable the documentation 
of production workflows, especially in cases where 3D data are 
hand-edited or reduced in some way through the production of 
scholarly outputs. These tools and schemata also should sup- 
port metadata capture for preservation and rights management 
purposes. 

e Astorage and retrieval platform for 3D data, designed to support 
the full range of 3D data uses, including creation, 3D printing, vi- 
sualization and analysis, and publication. 


Libraries can take the lead in supporting these 3D/ VR infra- 
structural components because they are the multidisciplinary hubs 
of academic institutions and have experience supporting the admin- 
istration of information resources, as well as research and develop- 
ment around new scholarly technologies. Furthermore, they harbor 
a long institutional history of preserving and providing access to 
knowledge for academic institutions, and they can meet the needs of 
multiple departments simultaneously. For this reason, many libraries 
are currently working to support 3D/VR projects through a combi- 
nation of providing training opportunities, such as workshops with- 
in the library, hiring staff to provide services in support of research 
projects, and working to create a communitywide framework for 
preservation. 

Several models of partnership may be forged among technolo- 
gists, faculty, and other interested stakeholders working toward the 
common goal of supporting current creation and preservation efforts 
using 3D / VR work in academic institutions. The diverse approaches 
discussed in these essays may guide librarians and digital curators 
alike as they approach the complex field of 3D/ VR data creation, 
dissemination, and preservation. Of course, each institution has a set 
of unique local needs; thus, each case study presented herein offers 
unique challenges and solutions that provide directions for further 
experimentation and research. At the same time, cross-disciplinary 
communities in the field (e.g., Community Standards for 3D Preser- 
vation, Building for Tomorrow, the Software Preservation Network, 
and Developing Library Strategy for 3D and Virtual Reality Collec- 
tion Development and Reuse, to name a few) are working toward a 
more cohesive model of addressing the standardization of preserva- 
tion methods and training; they hope to integrate their findings into 
a holistic approach that addresses the needs of communities across 
national and international contexts. As more libraries take up the 
call to action around this topic, we hope that this report—presented 
as a snapshot of the current state of the field—can provide an entrée 
into critical engagement with the exciting field of 3D / VR and other 
emerging scholarly tools in the rapidly changing academic library of 
the twenty-first century. 


The idea of a CLIR colloquium and proceedings report on 3D/ virtual reality 
(3D/VR) arose from early conversations among CLIR Postdoctoral Fellows 
interested in 3D/VR scholarship and the subsequent development of a work- 
ing group at the Coalition for Networked Information (CNI) in 2017. Through 
a generous microgrant from CLIR funded by the Alfred P. Sloan Foundation, 
we were able to realize our vision for a colloquium that would bring together 
diverse perspectives on this exciting and rapidly changing field. We are grate- 
ful to CLIR for its financial, logistical, and intellectual support throughout 
this process. In particular, we would like to thank Elliott Shore and Lauren 
Coats for encouraging these discussions by building into the CLIR education 
and training program formal opportunities to collaborate. We would also 
like to thank Alice Bishop and Christa Williford, who provided unwavering 
support and guidance throughout the application, planning, and publication 
process. Special thanks to Elizabeth Parke for her key input on colloquium 
planning and logistics, Diane Ramirez for her help with colloquium travel 
and reimbursements, and Kathlin Smith for her expert direction and astute 
input on report publication. 

The colloquium itself was truly a collaborative endeavor, and we are 
indebted to numerous sponsors and participants who made this event so 
successful. Thank you to our cosponsoring institutions—the University of 
Oklahoma Libraries; University Library at the University of California, Santa 
Cruz; and Temple University Libraries. The support of many individuals 
within these institutions, namely Tara Carlisle, Matt Cook, Carl Grant, and 
Joe Luccia, contributed greatly to our efforts. Thank you also to all of the at- 
tendees of the CLIR 3D/VR colloquium, who represented a range of perspec- 
tives and contributed to the enriching conversations that supplemented our 
scheduled presentations and shaped the content within these pages. 

We would also like to extend our appreciation to our peer reviewers, 
whose expertise and support improved this report immeasurably: Christy 
Caldwell, University of California, Santa Cruz; Jasmine Clark, Temple Uni- 
versity; Matt Cook, University of Oklahoma; Andrea Copeland, Indiana 
University—Purdue University, Indianapolis; Bill Endres, University of Okla- 
homa; Chad Hutchens, University of Wyoming; Tom Lee, University of Con- 
necticut; Kate Murray, Library of Congress; Elizabeth Parke, McGill Univer- 
sity; Samantha Porter, University of Minnesota; Anthony Sanchez, University 
of Arizona; Hannah Scates Kettler, University of Iowa; Carla Schroer, Cultural 
Heritage Imaging; Li Sou, University of Bradford; Chris Strasbaugh, Ohio 
State University; Edward Triplett, Duke University; and Jamie Wittenberg, 
Indiana University, Bloomington. Finally, thank you to Reed Sciven for the 
cover image design. 
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360-degree video: Digital video files created by using either special lenses 
or multiple cameras to capture all angles of view (i.e., images 360 degrees 
around the camera setup). For playback on a computer monitor, the user se- 
lects the part of the scene to be viewed by means of a keyboard and mouse, 
and then moves the perspective around the recorded 360-degree scene. With 
a VR headset, users move their heads and bodies to see different viewing 
angles of the scene. 


3D file formats: 3D meshes 


a. 


X3D: Open-source, royalty-free International Organization for 
Standardization—compliant standard that defines a 3D data file (encoded 
in a variety of formats, including XML, ClassicVRML, compressed bi- 
nary encoding, and JSON encoding). It replaced Virtual Reality Markup 
Language (VRML) in 2001. It is actively maintained by the Web3D 
Consortium. File extensions include .x3d, .x3dv, .x3db, .x3dz, .x3dbz, and 
x3dvz. 

OBJ: File format for encoding 3D geometry. Originally developed by 
Wavefront Technologies, it is now an open format and widely supported 
by 3D modeling software. OBJ files store only geometry information, so 
textures have to be stored in a separate file. OBJ files have the extension 
.obj. Because they are supported by most software programs, OBJ files are 
widely used—even though they lack a variety of functions, including in- 
ternal texture mapping and embedded metadata capabilities. 

DAE: Open-source, International Organization for Standardization— 
compliant standard file format for encoding 3D data. Also referred to as 
COLLAborative Design Activity (COLLADA), DAE has provision for 
some metadata fields, can contain scale information, and is encoded as 
XML. It was originally designed as an interchange standard for mov- 

ing 3D models between different 3D modeling and animation packages. 
Many 3D software packages can open and export DAE files. The nonprofit 
Khronos Group currently manages DAE. The file extension is .dae. 

PLY: File format developed at Stanford University by Greg Turk in 1994 to 
encode 3D geometry and some surface properties. It is also known as the 
Polygon File Format or, less commonly, the Stanford Triangle Format. The 
file extension is .ply. It has more functions than OBJ, but lacks the rich fea- 
ture set, including embedded metadata, of DAE and X3D. 

FBX: Proprietary 3D file format developed by Kaydara and now owned by 
the large CAD modeling software corporation, Autodesk. Derived from 
FilmBox, it is not openly documented, but it can be encoded as ASCII in a 
structured form that is human readable. The file extension is .fbx. 
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f. STL: File format used by CAD software to encode 3D information. The 
name is derived from its use in stereolithography, a form of 3D printing. It 
is often output from 3D modeling software in order to enable 3D printing. 
It stores geometric information in a 3D, Cartesian coordinate system and 
does not carry surface texture information or scale information. The file 
format is .stl. 


AR (augmented reality): Technologies that blend computer-generated im- 
ages, sounds, and haptic sensory inputs with real-world sensations. Unlike 
VR technologies, AR does not cover up the user’s sensory field completely, 
but “augments” it by adding additional layers of sensory information that 
complement real-world phenomena. 


assets: A general term for individual components that compose 3D and vir- 
tual environments (see: mesh). 


BIM (building information modeling): Use of digital tools to model the 
physical and functional properties of architectural spaces. It enables govern- 
ments and businesses to plan for and manage critical infrastructure. 


CAD (computer-aided design): Use of computers to assist in the creation 
of designs, including architectural and engineering designs. They can be 2D 
(taking the place of traditional drafting techniques) or 3D (taking the place 
of model building). Architectural and engineering design work is now done 
primarily using CAD. 


decimate: To reduce the information in a 3D polygon mesh by simplifying 
its geometry (i.e., by reducing the number of polygons). This is often neces- 
sary to make it more practical to work with the resulting mesh files, which 
can otherwise be very large. Web-based viewers and virtual reality systems, 
because of the limitations of graphics processor power, have limitations on 
the quantity of polygons that can be displayed at one time without system 
slowdown. 


GIS (geographic information system): System that displays, manages, and 
analyzes layers of spatial data. 


LiDAR (light detection and ranging): Remote-sensing technology that mea- 
sures distance using low-energy laser pulses. Often used in aerial surveys, 
LiDAR technology emits a laser pulse, which reflects off an object, and re- 
cords its return delay to determine distance points. Compiled distance points 
can be used to generate 3D representations of target objects (see point cloud). 


MR (mixed reality): Technologies that blend computer-generated images, 
sounds, and haptic sensory inputs with real-world sensations. MR is similar 
to AR in that digital layers of sensory input overlay real-world phenomena, 
yet MR technologies more convincingly “mix” virtual and physical reality 
through the real-time interaction between the two. 
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mesh: Generated from a set of data points (see point cloud) by software (e.g., 
Agisoft Photoscan and other photogrammetric processing software), a grid 
of triangles or other polygons that define the 3D surface of an object. Because 
the mesh is composed of geometric elements that have vertices (points) and 
lines, they can represent the same spatial information as a point cloud with 
fewer data points. In addition, this creates a “watertight” model that can be 
3D printed or brought into a VR environment and analyzed as a solid object 
rather than a set of discrete data points floating in space. Meshes can also be 
generated from computer-aided design (CAD) projects, in which there is no 
original source object. 


photogrammetry: Technique in which (at least) two photographs are taken 
from slightly different perspectives to calculate the three dimensional coor- 
dinates of a space or object. By measuring changes in the vertical (y axis) and 
horizontal (x axis) position of points in each photograph, the distance from 
the camera to the object in question can be calculated, producing the data on 
the depth (z axis). The basic technique dates back to the nineteenth century 
when surveyors and those producing topographical maps measured points 
in photographs and mechanically compared them to calculate precise spatial 
coordinates. Contemporary photography uses computers and sometimes 
hundreds of captured images to rapidly produce highly detailed 3D data that 
can be used to produce 3D models in the form of polygon meshes (see mesh). 


point cloud: Set of data points that describe the x, y, and z spatial coordinates 
of real-world objects. Obtained by light detection and ranging (LiDAR) and 
photogrammetry techniques, these data points can number in the millions. 
When rendered graphically, they resemble a cloud of individual points sus- 
pended in space. Depending on the quality of the scanning process, there 
may be holes in the data that need to be filled in before the point cloud can 
be converted into a polygon mesh (see mesh) for 3D modeling and printing 
purposes. 


retopologization: Process in which the surface of a 3D polygon mesh (see 
mesh) is replaced with a more efficient configuration of polygons. It can help 
reduce the file size and ensure that edges of polygons conform to the edges of 
the model features, which makes animation and other activities easier to ac- 
complish. Tools provided with 3D modeling software are used in the process. 


structured-light scanning: Highly accurate, 3D scanning process that in- 
volves superimposing patterns of light onto a physical object and a camera to 
capture information about the distortion of light patterns caused by contours 
of the object. Structured-light scanning can capture large sections of an object 
at once and can be quicker than some other capture processes, including pho- 
togrammetry and laser scanning. 


texture map / UV mapping: Map that carries color and texture information 
of an object and defines how that information should be wrapped around a 
3D mesh (see mesh). UV mapping is a common way of defining and attaching 
texture maps. 
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VR (virtual reality): Use of complex arrangements of computer software 

and hardware to re-create the sensory impression of real-world experience. 
Current systems use head-mounted displays (HMDs), hand controllers, and 
tracking sensors to produce interactive and immersive worlds composed 

of stereoscopic images and sounds. Some VR systems have started to pro- 
vide haptic and olfactory interfaces as well as sound and image. In addition 
to HMDs, CAVE systems (employing large, multiscreened rooms and 3D 
glasses with head tracking) have been developed, but they are far more costly 
than the current wave of commercially available HMDs and require custom 
installation. 


XR (extended reality): Overarching term encompassing the full spectrum of 
experience, from real-life to blended to full immersive reality (see 360-degree 
video, AR, MR, and VR). 
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